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PBEFACE 


This  report  describes  the  work  conducted  by  ARINC  Research  Corporation 
in  the  second  and  final  phase  of  research  into  field  testability  of  U.S. 
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F30602-84-C-0046  with  the  System  Reliability  and  Engineering  Branch  of  the 
Rome  Air  Development  Center.  This  work  results  from  the  contributions  of 
many  individuals  without  whom  the  final  analyses  would  be  less  than 
complete.  Major  contributions  were  made  by  the  following: 


'  Heather  Oussault.  RADC/RBET.  Griffiss  AFB 

-  HSgt.  Larry  Spencer.  1st  TACFITVING.  Langley  AFB 

-  TSgt.  Morris  Phinnesee.  436  AMS.  Dover  AFB 

-  TSgt.  Ron  Humphrey,  ASD/YPFF,  Uright  Patterson  AFB 

-  SSgt.  James  Jacintho,  1st  TACFITWING,  Langley  AFB 
'  Mr.  Don  Nichols.  MMIR  Branch  Office,  Kelly  AFB 


Accession  For 

NTIS  GRAtl 
DTIC  TAB 
Unannounced  '  □ 
Justification - 


By - 

Distribution/ 


Availability  Codes 


Dlst 


Avail  and/or 
SpeoiaJ. 


EXECUTIVE  SUMART 


INTRODUCTION 

A  pro>irai£  was  undertaken  to  develop  prediction  procedures  for 
organizations  testability  atti^butes  of  conplex  electronic 

equipment  used  ./y  the  Air  Force.  The  prediction  of  these  attributes 
during  design  phases  can  be  used  to  lacffiitor  and  improve  the  design  for 
field  maintainability  and  to  provide  an  early  indication  of  potential 
naintainabllity  problems.  This  report  describes  that  effort.  The  program 
was  conducted  in  two  phases  with  the  following  objectives: 

-  Phase  I 

—  Establish  definitions  and  mathematical  frameworks  necessary 
for  developing  prediction  procedures. 

—  Determine  the  feasibility  of  accomplishing  the  overall  objec¬ 
tives  of  de /eloping  prediction  equations  of  testability 
attributes. 

—  Identify  desigii  and  operational  characteristics  that  influence 
field  testability  ottrlbutes. 

-  Phase  II 

—  Develop  field  testability  attribute  prediction  equations  and 
techniques. 

—  Verify  these  prediction  techniques. 

—  Develop  guidelines  for  using  the  prediction  techniques. 
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Both  phases  concentrated  on  three  basic  descriptors  of  field 
maintenance: 

-  Fraction  of  faults  detected  (FFD) 

-  Fraction  of  faults  Isolated  (FFI) 

-  Fraction  of  false  alarms  (FFIi) 

A  fourth  descriptor,  false  alarm  rate  (FAR),  a  time-normalized  value  of 
FFA.  was  also  considered.  These  four  attributes  are  determined  to  be  the 
most  commonly  specified  attributes  for  field-level  testability. 

PHASE  I  RESULTS 
Introduction 

Definitions  and  mathematical  frameworks  related  to  organizational-level 
testability  were  developed  by  applying  three  approaches  to  modeling  the 
organizational-level  maintenance  process.  The  three  modeling  approaches 
employed  were: 

-  Set  Theory  Model  -  Based  on  using  Venn  diagrams  and  set-^oembership 
approaches  to  derive  definitions  and  algorithms. 

-  Modified  State  Model  -  Based  on  combining  actions  at  the 
organizational -maintenance  level  necessary  to  discover  the  system 
state  (e.g.,  failed  or  nonfailed). 

-  Flow  Model  -  Based  on  the  'flow  of  systems  and  subsyst^ns  through 
the  organizational-level  malncenance  process. 

The  Phase  I  report*  describes  the  models  and  algorithms  used  to 
develop  the  definitions  and  evaluation  procedures  for  the  three  testability 
ottributes:  fraction  of  faults  detected,  fraction  of  faults  isolated,  and 
fraction  of  false  alarms. 


♦Prediction  and  Analysis  of  Testability  Attributes:  Organizational -Level 
Testability  Prediction.  ARINC  Research  Corporation.  RADC-TR-85-268 , 

Feb  uary  1986.  AD#  A167957. 


Definitions 

A  key  objective  of  the  first  phase  of  the  program  was  to  develop 
accurate,  quantitative  definitions  of  the  three  lestablllty  attributes. 
The  definitions  were  to  be  relevant,  consistent  with  military  standards, 
mathematically  precise,  and  measurable.  It  would  be  of  little  use  to 
derive  a  set  of  equations  that  could  not  be  used  to  measure  fielded  system 
attributes.  The  following  definitions  were  developed  for  system-level 
fault  detection,  fault  Isolation,  and  false  alarms: 


-  Fcult  Detection  -  Normal  system  maintenance  (NSN)  Indicates  that 
the  syston  Is  not  functioning  properly,  and  this  Indication  is  the 
result  of  a  fault  within  the  system. 

-  Fault  Isolation  -  NSH  Identifies  all  failed  units  within  the 
system.  Fault  Isolation  may  be  either  proper  or  Improper. 

--  Proper  Fault  Isolation  -  only  and  all  failed  units  are 
Isolated. 

—  Improper  Fault  Isolation  -  All  but  not  only  failed  units  are 
Isolated. 

NOTE:  Any  other  outcome  of  an  attempted  Isolation  Is  con¬ 
sidered  to  result  in  no  fault  isolation. 

-  False  Alarm  -  There  Is  an  Indication  of  failure  In  the  system,  but 
there  is  no  failure  In  the  system.  False-alarm  rate  is  the  sum  of 
false  alarms  over  a  general  time  period  divided  by  that  time 
period. 


More  detailed  descriptions  of  these  and  other  deflnHions  used  in 
this  study  are  given  in  the  Phase  I  report. 


Phase  I  Feasibility  Summar^ 


The  feasibility  of  developing  prediction  procedures  for  the  three 
testability  attributes  was  determined  based  on  two  major  criteria;  (1)  the 
ability  to  measure  FFD,  FFI,  and  FFA  In  currently  fielded  systems  and 
(2)  the  ability  to  relate  specific  design  characteristics  to  measured 
values  of  the  testability  attributes.  If  these  criteria  were  satisfied, 
the  development  of  prediction  procedures  was  considered  feasible. 
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Field  measurement  of  the  three  testability  attributes  of  Interest  Is 
difficult.  The  current  Air  Force  maintenance  data  collection  system  does 
not  provide  direct  measures  of  FFD,  FFI.  or  FFA;  however,  it  does  record 
"cannot  duplicate"  events.  A  measurcntent  of  "cannot  duplicate"  events  and 
number  of  maintenance  actions  could  be  used  to  derive  a  measure  of  false 
al.)rins.  The  field  measurement  of  FFD  and  FFI.  however,  requires  direct 
observation  of  what  triggered  the  maintenance  activity  and  how  fault 
isolation  was  achieved.  Other  measures  of  FFD  and  FFI  could  be  derived 
using  system  design  information,  maintainability  demonstration  and  opera¬ 
tional  test  and  evaluation  data,  and  testability  modeling  and  analysis 
data. 

Establishing  relationships  between  system  design  characteristics  and 
the  testability  attributes  was  feasible  once  meas  ires  of  the  attributes  or 
surrogate  measures  were  obtained.  These  design  characteristics  Include 
number  of  elements,  number  of  test  points,  number  of  feedback  loops,  degree 
of  parallelism  In  the  design,  and  connector  dependency.  An  Investigation 
of  possible  relationships  between  the  design  characteristics  and  the 
testability  attributes  was  conducted  using  a  limited  data  set  and  only  one 
testability  attribute.  FFA.  The  preliminary  results  of  the  Investigation 
Indicated  that  a  relationship  exists  between  the  degree  of  parallelism  In 
a  given  design  and  the  number  of  false  alarms  experienced  by  the  system. 
In  general,  the  feasibility  of  developing  the  relationships  appeared  vo  be 
promising. 


The  continuation  of  the  work  Into  Phase  II  —  developing  prediction 
procedures  --  required  thac  the  difficulties  In  measuring  the  three  testa¬ 
bility  attributes  be  overcome.  Two  different  approaches  were  used  to 
obtain  the  Information  necessary  to  develop  measures  of  the  three  testa¬ 
bility  attributes.  First,  field  data  on  maintenance  actions  and  "cannot 
duplicate"  events  were  gathered  for  a  number  of  line  replacement  units 
(LRUs).  From  these  field  data,  measures  of  upper  limits  on  false  alarms 
were  then  determined  using  an  heuristic  approach.  The  result  was  that 


cannot  duplicate  (QD)  events  could  be  used  as  an  estimator  'upper  bound) 
of  false-alarm  events.  Second,  measures  of  fractions  of  faults  detected 
and  fraction  of  faults  Isolated  were  derived  from  the  application  of  the 
System  Testability  and  Maintenance  Program  (STAMP*)  analysis  model. 

STAMP*  Is  an  artificially  Intelligent,  computer-aided,  deslgn-for- 
testability  (OPT)  tool.  It  has  been  applied  to  a  large  number  of  systems, 
many  of  which  are  fielded.  The  results  of  STAMP*  have  been  found  to  be 
consistent  with  field  observations.  The  STAMP*  testability  imxlel  was 
chosen  because  of  Its  compatibility  with  the  project  objectives  and  the 
availability  of  prior  analyses  In-house  at  ARINC  Research  Corporation. 

STAMP*  determines  the  internal  Information  structure  of  systems  being 
analyzed  by  examining  the  dependency  topology  of  the  functions  within  the 
system.  As  such.  It  Is  able  to  map  the  Information  that  Is  or  Is  not 
available  at  certain  points  within  the  system.  The  following  two  measures 
were  chosen  as  surrogate  measures  of  field  testability  parameters: 


a.  Isolation  Level  (ID  -  Represents  the  number  of  Isolatlcm  conclu¬ 
sions  that  can  be  reached  by  the  Informaticm  flow  in  the  system. 
Isolation  conclusion  is  the  result  of  a  fault  Isolatlcm.*  that 
Is.  a  single  element  that  has  failed,  a  group  of  elements  that 
contain  the  failure,  or  no  fault  found  (sometimes  referred  to  as 
HTOK  In  STAMP*  nomenclature).  It  is  normalized  by  the  total 
number  of  conclusions  possible.  As  such.  It  represents  an  upper 
bound  on  FFI.  If  a  perfect  Implementation  were  possible,  the  II. 
and  FFI  should  approach  equality.  In  practice,  actual  test 
design  Implementation  will  not  use  the  complete  Information  flow 
In  the  systmn.  (Chapter  Foui  describes  this  parameter.) 

b.  Nondetection  Percentage  (NDP)  -  This  represents  the  percent  of 
Isolation  conclusions  for  which  there  Is  no  supporting  informa¬ 
tion  flow  and.  as  such,  gives  a  measure  of  nondetection.  The 
complement  of  this  measure  Is  detection  percentage  (DP*1-NDP)  and 
represents  the  percent  of  conclusions  for  which  there  Is  suffi¬ 
cient  testability.*  This  measure  represents  an  upper  bound  on 
FFD.  If  a  perfect  Implementation  were  possible,  the  two  should 
approach  equality.  (Chapter  Four  describes  this  parameter.) 


*For  definition  of  these  terms,  see  Appendix  D. 
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As  a  result  of  the  Phase  I  study,  the  measures  to  be  use<j  for  repre¬ 
senting  the  field  testability  characteristics  of  Interest  were  to  be 
developed  from  the  following  sources: 


Source 

Field  Data  [combined  organizational 
and  Intermediate  (O/I)  level]* 

Field  Data  (combined  O/I  level)* 

STAMPS  testability  analysis  param¬ 
eter  Isolatlm  level  (ID  used  as 
an  upper  Halt  to  FFI 

STAMPS  testability  analysis  param¬ 
eter  detection  percmtage  (DP) 
(complement  of  nondetectloi, 
percentage)  used  as  an  upper  Halt 
to  FFD 


Those  attributes  were  used  either  Individually  or  In  combination  to 
develoo  testability  characteristics  as  follows: 


CND  burden  «■  CND/NA  «  ffa  (represents  an  upper  Halt) 

CMD  rate  •  CND/ operating  hour  «  FAR  (represents  an  upper  limit) 
XL  s  ffI  (represents  an  upper  limit) 

1  -  NDP  £  FFD  (represents  an  upper  limit) 


These  estimators  were  the  focus  of  the  Phase  II  analysis. 


t 
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*The  merging  of  the  AFTO-349  data  for  the  O/I  maintenance  levels  led  to 
combining  them  for  field  data  analysis. 


PHASE  II 


Approach 

Phast  II  tachnlcal  objactlvaa  wara  to  provlda  tha  batlc  fora  and 
substanca  of  tha  pradlction  aquations.  Tha  approach  was  davalopad  through 
tha  following  tasks: 

-  Collact  data  on  a  significantly  larga  sanpla  of  systaas  and 
nalntananca  avants. 

‘  Davalop  a  thaoratlcal  basis  upon  «ihlch  to  build  ralatlonal 
aquations. 

-  Analyza  tha  collactad  data  through  ragrasslon  analysis  uslmj  tha 
davalopad  thaoratlcal  basis. 

-  Valldata  tha  pradlction  procaduras  by  application  to  ona  or  nora 
data  sats  not  usad  during  davalopaant. 

-  .develop  an  applications  oriented  approach  to  the  utilization  of 
these  predictor  equations. 

Results  —  Data  Base  Devalopaant 

APTO-349  field  data,  which  Included  CM)  Inforaatlcm.  were  collected 
for  36  LRUs  Installed  on  three  aircraft  over  ona  year  of  oparatlcms  (May 
1985  through  April  1986).  A  total  of  22.520  Mlntmance  actions  were 
tabulated.  In  addition,  extensive  coopllatlon  of  design  data  on  each  of 
the  38  LRUs  was  based  on  the  theoretical  analysis  of  functionality  and  the 
data  gathering  forms  developed  during  Mase  II. 

Testability  analyses  performed  by  STAMP*  were  available  for  22  sys¬ 
tems,  Including  one  system  (P-15  Radar.  AN/APG-63)  In  coanon  with  available 
field  data,  similar  extensive  design  data  ware  gathered  for  each  of  these 
systems.  The  STAMP*  data  were  supplemented  with  a  number  of  "synthesized* 
syst«ns  to  bring  the  total  to  35  for  the  analysis  of  detection  data. 

The  table  below  provides  a  summary  of  observational  data  used  In  the 
analyses. 
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SUMMARY  OF  OBSERVATIONS 

Testability 

Attribute 

Estimator 

Number  of 
Systems  L 

Number  of 
Field 

Observations 

FFA 

CND  burden 

38 

22.520 

FAR 

CND  rate 

38 

22,520 

FFI 

IL 

22 

* 

FFD 

DP 

35 

* 

*Not  measurable  by  current  reporting  systems. 


The  Analytical  Development  of  Relationships 

A  key  element  in  the  Phase  II  work  was  to  avoid  "blind"  regression  of 
masses  of  data.  The  objective  of  the  analytical  development  task  was 
twofold.  First,  identify  measurable  design  characteristics  to  be  used  in 
regression  analyses  wh^re  the  dependent  variables  are  fraction  of  false 
alarms,  fraction  of  faults  isolated,  and  fractions  of  faults  detected. 

Second,  anticipate  and  explain  the  results  of  the  regression  analyses 
through  coars^-scale  analytical  modeling. 

To  accomplish  these  goals,  causally  oriented  taxoncxnic  models  of 
cannot  duplicate  events  and  failures  were  developed.  These,  together  with 
literature  surveys  and  interviews  of  testability  and  maintenance  experts, 
were  used  to  develop  a  comprehensive  design  data  list.  This  ccxnprehenslve 
list  was  refined  through  several  iterations  into  the  system-level  design 
data  gathering  forms  described. 

Further,  analytical  expressions  were  developed  that  related  the 
dependent  variables  to  12  generic  design  characteristic  classes  shown  In 
the  list  below. 
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FUNC*fIONAL  REUTIONSHIPS  TO  EXAMINE 


Class 

CND  Rate  B  FAR 

CND  Burden  b  FFA  IL 

e  FFI 

DP  «  FFD 

Operational 

Complexity 

X 

X 

X 

X 

Topological 

Complexity 

X 

X 

X 

X 

Functional 

Complexity 

X 

X 

Environmental 

Factors 

X 

X 

X 

X 

Transient 

Factors 

X 

X 

X 

X 

Component 

Characteristics 

X 

X 

X 

X 

Numbers  of 
Components 

X 

X 

X 

X 

Accessibility 

X 

X 

X 

X 

Failure  Rate 

X 

X 

X 

Documentation 

Quality 

X 

X 

X 

X 

Topological 

Measures 

X 

X 

Test  Measures 

X 

X 

Finally,  many 

of  the  generic 

design  characteristics 

were 

developed 

Into  analytic  forms  to  aid  In  the  setup  of  the  regression  analysis. 


Prediction  and  Validation  Results 

Predictions  were  developed  for  each  of  the  four  parameters  previously 
discussed.  Their  functional  breakdown  is  presented  below. 
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CND  burden  =  f  (accessibility,  topological  complexity,  and  environ¬ 
mental  factors) 

CND  rate  =  f  (failure  rate,  transient  factors,  and  topological 
complexity) 

XL  £  ffi  =  f  (topological  measures,  test  measures,  and  ccxnponent 
characteristics) 

DP  «  FFD  f  (topological  measures,  test  measures,  ccxnponent 
characteristics,  and  a  nunu,.'r  of  others) 

These  functions  were  confirmed  by  the  analytical  model.  The  exact 
equations  are  presented  in  the  text  together  with  computation  and  measure- 
ment  procedures  necessary  to  make  actual  predictions. 

The  development  of  predictors,  as  guided  by  the  analytic  development 
of  functionality,  was  moderately  successful.  The  list  below  provides  a 
summary  of  prediction  correlations  and  validation  results. 


PREDICTION  CORRELATIONS  AMD  VALIDATION  RESULTS 


Testability 

Parameter 

r2 

Correlation 

Before 

Validation 

Validation 

Result 

r2 

Correlation 

After 

Validation 

Qualitative 
Evaluation  of 
Predictor 

CND  Burden 

0.63 

Adequate 

0.60 

Fair 

CND  Rote 

0.92 

Excellent 

0.91 

Excellent 

IL 

0.80 

Good 

0.80 

Good 

DP 

0.41 

Poor 

0.26 

Unuseable 

Validation  was  achieved  by  predicting  one  or  more  systems  that  did 
not  participate  in  the  data  base.  After  validation,  these  systms  were 
folded  back  in  to  improve  the  statistical  significance  of  the  data  base. 
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Conclusions  and  Recommendations 


This  research  yielded  a  number  of  useful  Insights  Into  field-level 
testability.  Tne  data  gathered  and  analyzed  during  this  project  represent 
a  reasonable  start  on  the  problem  of  prediction  of  field-level  testability 
attributes  for  complex  electronic  systems.  The  correlations  for  CND  rate 
(an  estimator  for  FAR)  and  IL  (an  estimator  for  FFI)  are  both  sufficient 
to  begin  predictive  work  during  design  phases.  The  validation  of  these 
prediction  equations  is  believed  to  be  sufficient  for  their  use  as 
estimators  for  design  compliance.  There  is  some  concern  that  a  CND  rate 
prediction  is  tied  to  the  accuracy  of  a  failure  rate  prediction. 

Prediction  of  CND  burden  is  marginal  at  this  time  for  design 
compliance,  but  may  be  useful  in  a  design  review  process  as  a  flag 
pointing  to  potential  problem  areas.  Detection  percentage  has  not  been 
well  enough  defined  (in  a  mathematical  sense)  for  use  at  this  time. 

The  following  actions  are  recommended: 


-  Develop  a  specification  procedure  for  field  CND  rate  as  opposed  to 
field  false-alarm  rate.  This  would  provide  a  specified  field 
testability  parameter  that  is  both  fleld-measureable  and 
predictable. 

-  Develop  a  specification  procedure  based  upon  the  isolation  level 
parameter  to  provide  a  specified  field  parameter  that  is 
predictable. 

-  Use  CND  rate  and  IL  predictions  during  preliminary  and  critical 
design  reviews  as  the  basis  for  verifying  or  requiring  manufacturer 
improvements  in  system  testability. 

-  Use  CND  burden  predictions  as  an  early  indicator  of  potential 
maintenance  problems. 
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CHAPTER  ONE 


INTRODUCTION  AND  BACKGROUND 


1.1  INTRODUCTION 

The  goal  of  the  research  performed  by  ARINC  Research  Corporation  was 
to  build  models  that  will,  in  some  measure,  predict  organizational-level 
testability  attributes  of  complex  electronic  equipment  on  the  basis  of 
design  characteristics.  Three  basic  attributes  of  the  organizational-level 
maintenance  system  were  considered: 

-  Fraction  of  faults  detected  (FFD) 

-  Fraction  of  faults  Isolated  (FFI) 

-  Fraction  of  false  alarms  (FFA) 

Oft 

False-alara  rate  (FAR) 

These  have  been  determined  to  be  the  most  conmonly  specified  attri¬ 
butes  of  field-level  testability.  The  project  was  performed  in  two 
phases,  each  with  its  own  technical  objectives.  Phase  I  technical 
objectives  were  to  provide  the  foundation  for  the  development  of  the 
predictor  model.  Phase  II  technical  objectives  were  to  provide  the  basic 
form  and  substance  of  predictive  equations. 
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1.1.1  Phase  1  Technical  Tasks 


Phase  I  was  conducted  through  the  following  tasks  and  separately 
^  1 

reported* 


‘  Survey  >he  current  literature  and  the  personnel  engaged  In 
organl:  :tlonal~level  maintenance.  . 

-  Compile  and  define  location  and  types  of  data  resources  currently 
available. 

-  Develop  a  consistent  mathematical  structure  that  will  permit  the 
measurement  of  the  required  parameters  and  the  development  of 
consistent  definitions. 

-  Determine  the  feasibility  of  developing  useful  prediction  methods 
and  Identify  the  approaches  necessary  for  such  development. 


1.1.2  Phase  II  Technical  Tasks 


Phase  II  was  conducted  through  the  following  tasks: 


-  Collect  data  on  a  significantly  large  sample  of  systems  and 
maintenance  events. 

-  Develop  a  theoretical  basis  upon  which  to  t»illd  relational 
equations. 

« 

-  Analyze  the  collected  data  through  regression  analysis  using  the 
developed  theoretical  basis. 

~  Validate  the  prediction  procedures  by  application  to  one  or  more 
data  sets  not  used  In  the  above-mentlrmed  development. 

-  Develop  an  applications  approach  to  the  utilization  of  these 
prediction  equations. 


These  tasks  are  described  In  this  report. 


J^Predlctlon  and  Analysis  of  Testability  Attributes:  Organizational- 
Level  Testability  Prediction.  W.  R.  Simpson,  J.  H.  Bailey,  K.  B.  Barto, 
and  E.  Esker,  RADC-TR-85-268,  Phase  I  Report,  Rome  Air  Development 
Center,  Grlffiss  AFB,  New  York,  February  1986. 


1.2  BI^CKGROUND 


As  a  result  of  Increased  system  complexity  and  sophistication,  the 
Mlntenance  of  electronic  systems  Is  becoming  more  difficult  and  costly, 
despite  advances  in  automatic  test  equipment.  '  Testability  design  Is 
usually  approached  from  the  bottom  up.  with  component  and  board  testa¬ 
bility  deslgn«.d  In,  but  with  little  attention  given  to  Isolation  of  the 
Individual  unit  In  the  full  system.  Current  design  of  systems  and  tests 
frequently  results  In  long  test  times  and  high  ambiguity  levels  for  fault 
Isolation.  False-alarm  and  "retest-OK”  (RTOK)  rates  of  40  percent  and 

greater  are  not  uncommon  In  many  avionic  systems.  Studies  of  the  F-16 
4  5 

aircraft  and  the  CH-54  helicopter  have  shown  that  troubleshooting 
can  consume  50  percent  or  more  of  the  total  man-hours  expended  on  repair. 
Av;ionlcs  Maintenance  Conference  reliability  reporting  statistics  Indicate 
similar  trends  In  avionics  repairs  for  the  scheduled  air  carriers.^ 

1.2.1  The  Testability  Discipline 

Testability  is  coming  to  be  recognized  as  a  valid  and  useful 
engineering  discipline.  The  recent  publication  of  a  testability 


^George  W.  Neuman.  Testing  Technology  Working  Group  Report  (IDA/OSD  R&M 
Study) .  Institute  for  Defense  Analysis,  IDA  D-41,  August  1983. 

^Vllllam  L.  Klener  and  Anthony  Coppola.  "Joint  Services  Program  In 
Design  for  Testability,"  Proceedings,  Annual  Reliability  and  Maintaina¬ 
bility  Symposium.  1981,  pg.  268. 

^Special  Report  on  Operational  Suitability  (05)  Verification  Study 
Focus  on  Maintainability.  M.  L.  Labik,  G.  T.  Harrison,  and  B.  L.  Retterer, 
ARIKIC  Research  Corporation.  Report  1751-01-2-2395,  February  1981. 

^Thomas  N.  Cook  and  John  Arlano,  "Analysis  of  Fault  Isolation 
Criteria/Techniques."  Proceedings,  Annual  Reliability  and  Maintainability 
Symposium.  1980.  pg.  29. 

^Avionics  Maintenance  Conference  Report.  AMC  Publication  84-083/MOF-28, 

Nay  25,  1984. 
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Standard  Is  svldenca  of  th*  Increasing  Importance  of  testability  in  the 
development  of  military  systems.  A  system  has  good  testability  when 
existing  faults  can  be  confidently  and  efficiently  identified.  Confidence 
is  achieved  by  frequently  and  unambiguously  idmtifying  only  the  failed 
components  or  parts,  with  no  removals  of  good  items  and  with  minimum  loss 
of  time  due  to  false  indications  or  false  alarms.  Efficiency  Is  achieved 
by  minimizing  the  resources  required,  such  as  man-hours,  test  equipment, 
and  training. 

1.2.2  Testability  as  a  Design  Variable 

The  number  of  tests  and  the  information  ccmtent  of  test  results. 

together  with  the  location  and  accessibility  of  test  points,  define  the 

testability  potential  of  an  equipment.  Testability  is.  of  course,  a 

design-related  characteristic.  There  are  few  standardized  tools  for  the 

evaluation  of  design  testability,  particularly  at  the  organizational 

level.  In  fact,  a  review  of  the  current  literature  suggests  that  even 

conmc  ;  definitions  of  testability  are  hard  to  find.  For  example. 

8 

Malcolm  states  that  built-in-test  (BIT)  false  alarms  can  be  one  of  two 

types:  a  BIT  indication  %ihen  there  are  no  faults  and  a  BIT  indication 

g 

when  the  fault  is  in  another  unit.  MIL-STD-1309  defines  a  false  alarm 
as  a  fault  Indication  where  no  fault  exists.  Whether  these  two  definitions 
are  consistent  depends  on  individual  Interpretation. 

For  testability  to  be  appropriately  and  consistently  incorporated 
into  the  design  process,  standard  definitions,  procedures,  and  tools  must 
be  developed  to  evaluate  and  predict  organizational-level  testability 


^Testability  Program  for  Electronic  Systems  and  Equipments.  MIL-STD-2165. 

26  January  1985. 

^J.  G.  Malcolm,  "BIT  False  Alarms:  An  Important  Factor  in  Operational 
Readiness."  Proceedings,  Annual  Reliability  and  Maintainability  Symposium. 
1982,  pg.  206. 

^Definitions  of  Terms  for  Test,  Measurement,  and  Diagnostic  Equipment. 
MI1.-STD-1309,  30  May  1975. 


attributes.  A  tesitabllity  evaluation  should  provide  not  only  predictions 
but  also  applicable  redesign  information  when  testability  attributes  are 
predicted  to  t/a  below  desired  levels. 

¥ 

1  2.3  Testability  and  Oroanlzational-Level  Maintenance 

It  is  at  the  organizational  level  that  system  faults  are  first 
detectec.  In  many  instances,  the  organizational-level  maintenance  is 
collocated  with  and  co-reported  with  the  intermediate-level  maintenance. 
The  interaction  of  subsystems  complicates  fault  identification  and  detec¬ 
tion.  Organizational-level  testability  is  a  primary  influence  on  mission 
readiness,  and  lack  of  fault  detection  at  this  level  can  lead  to  mission 
failure.  Of  the  many  testability  attributes  explored,  three  are  directly 
related  to  the  ability  of  complex  electronic  syst^ns  to  meet  mission 
requl  rements : 


-  Fraction  of  Faults  Detected  -  Ideally.  FFD  should  be  100  percent. 
Any  fault  not  detected  prior  to  a  mission,  either  by  BIT.  built-in¬ 
test  equipment  (BITE),  or  by  maintenance  operations  ready 
(OPSREAOY)  test,  could  result  in  a  failed  or  aborted  mission. 
Further,  if  the  failure  is  not  detected  during  or  after  completion 
of  the  mission,  the  following  mission  could  be  jeopardized.  In 
reality,  some  system  faults  are  less  critical  than  others,  and  an 
FFD  smaller  than  100  percent  might  be  tolerable. 

-  Fraction  of  Faults  Isolated  -  The  ideal  value  of  FFI  is  100  per¬ 
cent.  If  a  detected  failure  is  not  Isolated  quickly  and  effi¬ 
ciently,  the  system  may  not  be  mission-ready  for  a  long  time.  To 
meet  the  mission-ready  requl r^nent,  maintenance  crews  may  "change 
out"  entire  mission-critical  systems  or  spend  a  great  deal  of  time 
using  "shotgun"  maintenance  approaches.  These  practices  compli¬ 
cate  already  difficult  sparing  and  logistics  problems  and  add  to 
system  llfs-cycle  costs.  Measures  associated  with  FFI  are  mean 
time  to  fault  Isolate  (MTFI)  and  mean  time  to  repair  (MTTR),  as 
well  as  ambiguity  group  statistics  and  RTOK  rates. 

-  Fraction  of  False  Ale.rms  -  The  ideal  value  of  FFA  is  zero;  FFA  is 
a  complementary  factor  of  FFD.  When  BIT/BTTE  or  OPSREADY  checks 
Indicate  failures  that  cannot  be  duplicated  or  Isolated  because 
they  do  not  exist,  the  sy5tT#ra  is  held  in  a  pre-mission-ready 
status  while  checkc  are  run  and  rerun.  A  high  FFA,  like  a  low 
FFI,  leads  to  system  change-outs  or  shotgun  maintenance  approaches. 
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-  False  Alarm  Rate  -  The  rate  of  occurrence  of  false  alarms.  The 
ideal  value  is  zero.  It  is  computed  as  a  time-normalized  sum  of 
false  alarms,  where  the  time  normalization  is  either  calendar  or 
operating  hours. 

1.3  REPORT  ORGANIZATION 

Chapter  Two  of  this  report  provides  a  brief  overview  of  the  Miase  I 
work  and  its  impact  on  the  Phase  II  program.  Chapter  Three  details  the 
analytical  background  to  be  applied  to  both  the  data  gathering  and  equa¬ 
tion  development.  Chapter  Four  presents  a  detailed  summary  of  the  data 
gathering  efforts  and  the  data  used  during  the  analysis. 

Chapter  Five  is  an  explanation  of.  the  data  analysis  and  regressions. 
Including  the  prediction  results  and  their  verification.  Chapter  Six 
includes  the  recommended  application  of  these  predictors.  It  also  Includes 
the  information  necessary  to  compute  all  of  the  variables  necessary  to 
apply  these  equations.  Our  conclusions  and  summary  are  presented  in 
Chapter  Seven. 

Appendixes  A  through  E  include  comprehensive  design  data  lists,  data 
gathering  forms,  regression  variables,  a  glossary,  references,  and 
bibliography. 
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CHAPTER  TWO 


OVERVIEW 


2.1  REVIEW  OP  PHASE  I 

Complex  interactions  occurred  between  work  performed  in  Phases  I  2md 
II.  These  interactions  affected  the  ground  rules  and  structure  of  Phase 

II.  The  purpose  of  Phase  I  was  to  structure  the  problem  of  field-level 

testability  prediction  so  that  a  consistent  math^natical  basis  could  be 
used  in  the  development  of  prediction  techniques.  A  key  elanent  was  in 
the  defined  breakdown  of  the  primary  analysis  parameters  of  FFD,  FFI.  and 
FFA. 


The  definitions  and  mathematical  frameworks  «iere  developed  through 
the  application  of  three  different  approaches  to  modeling  the 
organizational-level  maintenance  process: 


-  Set  Theory  Model  -  Model  was  developed  through  the  use  of  Venn 
diagrams  and  set-membership  approaches  to  derive  definitions  and 
algorithms. 

-  Modified  State  Model  -  Model  based  on  the  combination  of  actions 
at  the  organizational-maintenance  level  necessary  to  discover  the 
system  state  (e.g..  failed  or  nonfailed). 

-  Flow  Model  -  Model  that  traces  the  flow  of  syst^ns  and  subsystems 
through  the  organizational  level  maintenance  process. 


The  use  of  the  different  approaches  had  several  advantages;  two  of 
these  advantages  were  the  following: 


-  The  insights  and  visibility  into  the  interpretation,  make-up.  and 
logical  content  of  a  testability  attribute  afforded  through  the 
use  of  one  modeling  approach  were  often  superior  to  those  provided 
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by  another.  Further,  the  different  viewpoints  provided  by  each  of 
the  modeling  approaches  combined  tc  provide  Insights  into  the  form 
and  content  of  the  attributes  that  could  not  have  been  provided  by 
the  application  of  a  single  model. 

-  The  use  of  the  three  modeling  approaches  provided  a  means  for 
crosschecking  the  results  of  the  models.  Because  all  three 
approaches  model  the  organizational -level  maintenance  process,  the 
three  models  must  provide  consistent  results. 


The  Phase  I  report  describes  the  models  and  algorithms  used  to  develop 
the  definitions  and  evaluation  procedures  for  the  three  testability  attri¬ 
butes,  as  well  as  data  sources,  definitions,  and  feasibility  of  developing 
predictors. 


The  feasibility  of  developing  pr^lction  procedures  for  the  three 
testability  attributes  was  determined  based  upon  two  major  criteria: 

(1)  the  ability  to  measure  FFD,  FFI,  and  FFA  in  currently  fielded  systems 
and  (2)  the  ability  to  relate  specific  design  parameters  to  measured 
values  of  the  testability  attributes.  If  these  criteria  could  be  satis¬ 
fied,  the  development  of  prediction  procedures  would  be  considered 
feasible. 

Field  measurement  of  the  three  testability  attributes  of  interest  is 
difficult.  The  current  Air  Force  maintenance  data  collection  system  does 
not  provide  direct  measures  of  FFD,  FFI,  or  FFA.  The  maintenance  data 
collection  system  does  record  cannot  duplicate  (CND)  events,  and  a 
measurement  of  CND  events  and  maintenance  actions  could  be  used  to  derive 
at  least  an  upper  limit  on  false  alarms.  The  field  measurement  of  FFD  and 
FFI,  however,  requires  direct  observation  of  what  triggered  the  maintenance 
activity  and  how  fault  isolation  was  achieved.  Other  measures  of  FFD  and 
FFI  can  be  derived  using  system  design  information,  maintainability 
demonstration  and  operational  test  and  evaluation  data,  and  testability 
modeling  and  analysis  data. 
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Establishing  relationships  between  system  design  characteristics  and 
the  testability  attributes  should  be  feasible  once  measures  of  the  attri¬ 
butes  can  be  obtained.  These  characteristics  include  number  of  elements, 
number  of  test  points,  number  of  feedback  loops,  degree  of  parallelism  in 
the  design,  and  connector  dependency.  An  investigation  of  possible  rela¬ 
tionships  between  the  design  characteristics  and  the  testability  attri¬ 
butes  was  conducted  in  Phase  I  using  a  limited  data  set  and  only  one 
testability  attribute.  FFA.  The  preliminary  results  of  the  investigation 
indicate  that  a  relationship  exists  between  the  degree  of  parallelism  in  a 
given  design  and  the  number  of  false  alarms  experienced  by  the  system.  In 
general,  the  feasibility  of  developing  the  relationships  appeared  to  be 
prcxnislng. 

2.2  PHASE  I  IMPLICATIONS  TO  PHASE  II 

The  continuation  of  the  Phase  II  work  toward  developing  prediction 
procedures  required  that  the  difficulties  in  measuring  the  three  testabil¬ 
ity  attributes  be  overcome.  Two  approaches  were  used  to  obtain  the  infor¬ 
mation  necessary  to  develop  measures  of  the  three  testability  attributes. 
First,  field  data  on  maintenance  actions  and  CND  events  were  gathered  for 
a  number  of  line  replaceable  units  (LRUs).  Measures  of  upper  limits  on 
false  alarms  were  then  obtained  from  this  field  data.  The  result  was  that 
we  us€fd  CND  events  as  an  estimator  (upper  bound)  of  false-alarm  events. 
Second,  measures  of  fractions  of  faults  detected  and  fraction  of  faults 
isolated  were  derived  from  the  application  of  the  testability  analysis 
model  System  Testability  and  Maintenance  Program  f STAMP® ) . 

STAMP®  is  an  artificially  intelligent,  computer-aided,  deslgn-for- 
testability  tool.^^  It  has  been  applied  to  a  large  number  of  systems, 
many  of  which  are  fielded.  The  results  of  STAMP®  have  been  found  to  be 


R.  Simpson,  "Stamp  Testability  and  Fault-Isolation  Applications, 
1981-1984,"  Proceedings  of  IEEE  1985  Autotestcon.  Long  Island,  New  York, 
October  1985. 
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consistent  with  field  observations.^^  The  STAMP*  testability  model  was 
chosen  because  of  its  conpatibility  with  the  project  objectives  and  the 
availability  of  prior  analyses  performed  at  ARINC  Research  Corporation. 


Two  measures  within  STAMP*  —  Isolation  level  (ID  and  nondetection 
percentage  (NDP)  —  were  chosen  as  surrogate  measures  of  field  testability 
parameters.  STAMP*  determines  the  Internal  information  structure  of  sys¬ 
tems  being  analyzed  by  examining  the  dependency  topology  of  the  functions 
within  the  system.  As  such,  it  is  able  to  map  the  information  that  is  or 
is  not  available  nt  certain  points  within  the  system.  The  XL  and  NDP 
represent  the  following: 


-  Isolation  Level  -  IL  represents  the  number  of  Isolation  conclusions 
that  can  be  reached  by  the  information  flow  in  the  systan.  It  is 
normalized  by  the  total  number  of  conclusions  possible.  As  such, 
it  represents  an  upper  bound  on  PFI.  This  upper  bound  assumes  a 
uniform  failure  probability.  If  the  factor  were  weighted  for 
falluc®  rate,  it  would  be  almost  identical  to  a  field-measured  FFI 
potential-  Perfect  implementation  and  relative  failure  rate 
weighting  w.>uld  be  essential  for  Isolation  level  and  fraction  of 
faults  isolatable  to  approach  equality.  In  practice,  actual  test 
design  Implementation  will  not  use  the  complete  information  flow 
in  the  system  (see  Chapter  Four  for  additional  discussion  of  this 
parameter) . 

-  Nondetection  Percentage  -  NDP  represents  the  percent  of  conclu¬ 
sions  for  which  there  is  no  supporting  information  flow  and,  as 
such,  gives  a  measure  of  nondetection.  The  complement  of  this 
measure  is  detection  percentage  (DP=1-NDP)  and  represents  the 
percent  of  conclusions  for  which  there  is  supporting  information 
flow.  This  measure  represents  an  upper  bound  on  FFD.  If  a 
perfect  Implementation  were  possible,  the  two  should  approach 
equality.  (See  Chapter  Four  for  additional  discussion  of  this 
parameter. ) 


As  a  result  of  the  data  base  used  for  developing  the  predictor  equa¬ 
tions,  the  Phase  II  work  involves  field  estimators  that  represent  upper 


R.  Simpson,  and  J.  R.  Agre,  "Experiences  Gained  in  Testability 
Design  Tradeoffs."  Proceedings  of  the  1984  IEEE  Autotestcon  Conference, 
Washington,  D.C.,  November  1984. 


limits  of  the  desired  parameters, 
following: 


These  synthesized  measures  are  the 


Testability  Attribute 

-  CND  (Cannot  Duplicate)  Events 

-  MA  (Maintenance  Actions) 

-  FFI 

-  FFD 


Source 

Field  Data  (combined  O/I  level)* 

Field  Data  (combined  O/I  level)* 

STAMP*  Testability  Analysis 
Parameter  Isolation  Level  used 
as  an  upper  limit  to  FFI 

STAMP*  Testability  Analysis 
Parameter  Detection  Percentage 
(Conplement  of  nondetection 
percentage)  used  as  an  upper 
limit  to  FFD 


The  attributes  were  used  either  Individually  or  In  combination  to 
develop  testability  characteristics  as  follows: 


CND  burden  »  CND/NA  s  ffa  (represents  an  upper  limit) 

CND  rate  >  CND/operatlng  hour  s  FAR  (represents  an  upper  limit) 

IL  sc  FFI  (represents  an  upper  limit) 

1  -  NDP  s  FFD  (represents  an  upper  limit) 

These  estimators  were  then  the  focus  of  the  Phase  II  analysis  as  described 
In  the  balance  of  this  report. 


*The  merging  of  the  AFTO-349  data  for  the  organizational  and  Intermediate 
(O/I)  maintenance  levels  led  to  combining  them  for  field  data  analysis. 
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CHAPTER  THREE 


ANALYTICAL  DEVELOPMENT 


3.1  INTRODUCTION 

The  objective  of  the  analytical  development  task  was  twofold.  First, 
Identify  measurable  design  characteristics  to  be  used  in  regression 
analyses  tihere  the  dependent  variables  are  fraction  of  false  alarms, 
fraction  of  faults  Isolated,  and  fraction  of  faults  detected.  Second, 
anticipate  or  explain  the  results  of  the  regression  analyses  through 
coarsf*  scale  analytical  modeling. 

One  of  the  conclusions  of  Phase  I  of  this  program  was  that  the 
testability  attributes  FFA,  FFI,  and  FFD  were  not  compatible  with  current 
Air  Force  maintenance  reporting  systems  and  are.  therefore,  not  readily 
obtainable  Consequently,  these  attributes  were  replaced  by  surrogate 
measures  for  the  purposes  of  completing  Phase  II  of  this  program.  Cannot 
duplic  a  ovents.  isolation  levels,  and  detection  percentages  were 
substltu.  .  for  FFA,  FFI.  and  FFD.  respectively.  (See  Chapter  Two  for 
explanati;....s  of  the  choice  of  these  surrogates  and  their  relationships  to 
the  original  testability  attributes.) 

As  th^  'riglnal  testability  attributes  were  replaced  by  surrogate 
measures,  so  was  the  approach  of  the  analytic  development  task.  Iden¬ 
tification  of  design  characteristics  was  undertaken  to  provide  the  neces¬ 
sary  data  for  statistical  regressions  on  the  surrogate  attributes.  The 
most  significant  Impact  of  this  modification  was  in  the  regression  on 
CND.  Whereas  IL  and  DP  approximate  FFI  and  FFD,  respectively,  FA  is  a 
proper  subset  of  CND.^  The  net  result  was  that  CND,  FFI,  and  FFD  were 
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used  as  the  basis  for  design  characteristic  identification  and  for  the 
subsequent  analytical  modeling. 

This  task  was  accomplished  in  three  steps:  unrestricted  design 
characteristic  identification,  selection  of  measurable  characteristics, 
and  Interactive  refinement  during  regression  analyses.  In  the  sections 
that  follow,  these  steps  and  their  results  are  described. 

3.2  UNRESTRICTED  DESIGN  CHARACTERISTIC  IDENTIFICATION 

The  process  of  identifying  design  characteristics  that  can  affect  the 
testability  attributes  CND.  FFI,  and  FFD  began  with  a  study  of  the  models 
for  organizational  maintenance  developed  during  Phase  I  of  this  effort. 
Particular  attention  was  paid  to  the  Set  Theory  Model. ^  This  model 
served  as  the  basis  for  the  development  of  a  causally  oriented  taxonomy  of 
CND  (Figure  3-1)  and  a  maintenance-oriented  classification  of  failure 
events  (Figure  3-2).  These  classifications  are  discussed  in  detail  in 
Section  3.4. 

A  review  of  pertinent  literature,  including  textbooks,  published 
reports,  technical  journals,  and  appropriate  military  documents,  was 
undertaken.  A  list  of  the  literature  reviewed  In  this  task  is  included  in 
the  bibliography  In  Appendix  D.  Also,  Interviews  were  conducted  with 
maintenance  and  reliability  experts  in  the  Air  Force  and  at  ARINC  Research 
The  results  of  those  investigations  were  first  used  to  refine  and  verify 
the  taxonomic  models  described  above.  Then,  using  t.he  models  as  a  basis, 
further  investigations  produced  an  unrestricted  list  of  design  charac¬ 
teristics  ("wish  list")  that  the  research  indicated  might  affect  the 
testability  attributes.  The  goal  was  to  ensure  that  we  would  have  a  super 
set  of  the  characteristics  necessary  for  successful  regression  analyses. 
This  “wish  list"  of  design  characteristics  is  given  in  Appendix  A. 


(The  appropriate  dlchotoales  peinit  the  separation  of  fraction  of  faults  detect< 
and  fraction  of  faults  Isolated.  Note:  branch  identified  "Incorrectly  Isolat 
Failures"  represents  an  area  of  overlap  with  cannot  duplicate  events). 


3.3  SELECTION  OF  MEASURABLE  DESIGN  CHARACTERISTICS 


The  list  In  Appendix  A  Is  a  set  of  characteristics  meaningful  for 
prediction  of  testability  attributes  but  not  necessarily  attainable  within 
the  scope  of  this  project.  A  number  of  compromises  were  made  to  the  data 
set  based  upon  the  following  criteria: 


-  Attainable  Measures  -  In  places  where  levels  of  detail  were  beyond 
the  engineering  labor  scope  or  simply  not  easily  available,  mean¬ 
ingful  combined  measures  were  sought  (see  discussion  below). 
Furthermore,  each  of  the  measures  should  be  attainable  without 
special  structuring  of  the  analysis.  For  example,  the  theoretical 
workup  Indicated  that  sneak  circuit  parameters  may  indeed  be 
Important  to  the  prediction  of  field  CND.  None  of  the  systems 
chosen  were  analyzed  for  sneak  circuits  because  such  an  analysis 
was  well  beyond  the  scope  of-  this  project.  However,  certain 
internal  characteristics  of  LRUs,  such  as  SRU-LRU  Interconnects, 
were  available  in  the  documentation,  and  these  were  acquired  In 
hopes  of  developing  some  sneak  circuit -related  parameters. 

-  Universality  of  Measures  -  Measures  that  applied  to  a  specific 
small  subset  of  the  data,  but  were  not  applicable  to  a  larger 
class  of  systems  or  were  not  attainable  in  a  statistically 
significant  number  of  different  electronic  systems,  were  not 
sought . 

-  Prediction  Applicability  -  A  design  measure  should  be  quantifiable 
by  a  testability  analyst  at  the  time  a  prediction  Is  normally 
performed. 


Our  Initial  goal  was  to  have  a  set  of  design  characteristics  for  each 
system  such  that  an  Individual  could  acquire  all  of  the  necessary  data 
within  two  to  three  hours  by  reviewing  technical  orders  and  interviewing 

Air  Force  maintenance  personnel.  In  practice,  the  actual  data  acquisition 

/ 

efforts  varied  between  1  hour  and  16  hours  for  each  LRU  examined. 

A  number  of  documentation  sets  (i.e..  Technical  Orders)  were  reviewed 
ar;j  analyzed.  As  a  consequence,  the  level  of  detail  Inherent  in  the 
design  characteristic  "wish  list"  had  to  be  reduced.  For  example,  the 
unrestricted  set  of  design  characteristics  required  information 
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descriptive  of  such  Items  as  the  logic  family,  level  of  Integration, 
bandwidth,  and  packaging  for  each  Integrated  circuit  In  a  system  under 
anal\rsls.  Because  Air  Force  Technical  Orders  typically  only  Identify  a 
component  as  being  an  Integrated  circuit,  only  the  numbers  of  Integrated 
clrcu:\ts  In  a  given  design  were  deemed  to  be  available  characteristics  for 
this  study.  A  standard  work  form  was  prepared  and,  subsequently,  used  to 
gather  design  characteristic  data  during  field  trips.  This  work  form  of 
measurable  design  characteristics  Is  ^own  In  Appendix  B. 

3.4  INTERACTIVE  REFINEMENT  OF  DESIGN  CHARACTERISTICS 

The  goal  of  this  subtask  was  twofold.  First,  prior  to  the  regression 
analysis  task,  coarsely  forecast  the  dependency  of  the  testability  attrlb* 
utes  on  ti'ie  various  measured  and  synthesized  design  characteristics.  As 
such,  this  required  synthesizing  appropriate  characteristics.  A  synthe¬ 
sized  characteristic  Is  derived  from  measured  characteristics.  It  is  used 
to  describe*  some  behavioral  aspect  of  a  design  that  Is  not  directly 
measurable  due  to  latis  of  physics  or  resource  limitations.  A  second  goal 
of  this  subtask  was  (as  the  regression  analysis  progressed)  to  Justify  any 
unexpected  findings  by  incorporating  the  results  In  the  analytical  model. 
Due  to  a  lack  of  resolution  Imposed  by  the  coarse  scale  of  the  design 
characteristics  measured,  the  analytical  model  was  restricted  to  basic, 
first-order  relationships.  The  subsections  that  follow  explain  the 
resulting  model . 

3.4.1  Taxonomic  Fk)dels 

3 . 4 . 1 . 1  Tl>e  Taxonomic  Model  of  CND  Events 

The  Phase  I  modeling  effort  sho%fed  that  all  of  the  testability 
attributes  In  question  are  Intimately  tied  to  the  field  CND  problem.  For 
example.  In  each  of  the  three  models  developed  during  Phase  I,  CND  appears 
in  every  one  of  the  testability  attributes  examined  (FFD,  FFI,  FFA,  and 
FAR).  To  that  end.  the  taxonomic  model  of  CND  was  developed  In  the 
greatest  detail.  Each  of  the  parameters  developed  for  CND  will  affect  all 
of  the  other  attributes,  and  the  functionality  of  any  parameter  that 


appears  In  CND  may  be  expected  to  appear  in  each  of  FFI  and  KFD.  The 
basic  taxonomic  model  of  CND  events  that  will  yield  parameters  for  all  of 
the  testability  attributes  Is  given  in  Figure  3-1.  As  shown  In  the  figure, 
CND  can  have  12  constituent  subclasses.  5  of  which  represent  CM)  events 
that  are  false  alarms  by  the  given  definitions  of  Phase  I  work.  The  12 
constituent  subclasses  of  CND  are  the  following: 


1. *  Operator  Error  -  The  operator  of  the  system  containing  the  unit 

under  test  Incorrectly  used  the  unit.  Incorrectly  Interpreted 
unit  behavior,  or  both:  the  operator  erroneously  percel>^d  and 
reported  a  malfunction,  and  no  malfunction  subsequwitly  can  be 
duplicated  by  maintenance  personnel. 

2. *  Latent  Built-In  Test  Design  Error  Hanlfestatlon  -  As  a  product 

of  coincidence,  an  appropriate  sequence  of  events  occurs  that 
causes  a  latent  BIT  design  error  to  manifest  Itself;  maintenance 
personnel  subsequently  cannot  duplicate  the  sequence  of  evmts 
that  precipitates  the  error  manifestation. 

3.  Environmentally  Induced  BIT  Error  -  Environmental  conditions, 
such  as  vibrations,  pressure,  and  temperature,  cause  transient 
behavior  In  the  BIT  system  such  that  a  malfunctl(x:  Is  erroneously 
reported:  maintenance  personnel  subsequently  cannot  reproduce 
the  conditions  that  caused  the  translmt  behavior. 

4.  BIT  Transient  Failure  -  Comp<»ent  degradation  In  the  BIT  subsys¬ 
tem  causes  a  failure  of  a  transient  nature,  resulting  in  an 
erroneous  report  of  a  malfunction  In  the  host  system,  and  the 
transient  behavior  subsequently  Is  not  exhibited  during  testing 
by  maintenance  personnel. 

5.  BIT  Hard  Failure  -  A  failure  occurs  In  a  BIT  subsystem,  a  mal¬ 
function  of  a  system  Is  reported,  and  the  suspect  system  Is  not 
host  to  the  accusing  BIT;  maintenance  personnel  subsequently 
verify  the  unit  to  be  good. 

6. ^  Latent  Design  Error  Nanlfestatlm  -  As  a  product  of  coincidence. 

an  appropriate  sequence  of  events  occurs  that  causes  a  latent 
design  error  In  a  system  to  manifest  Itself;  subsequently, 
maintenance  personnel  cannot  duplicate  the  sequence  of  events 
that  precipitates  the  error  manlfestatlcm. 


^Subclass  Is  a  false  alarm  as  denoted  in  Prediction  and  Analysis  of  Test¬ 
ability  Attributes:  Organizational-Level  Testability  Prediction.  W.  R. 
Simpson.  J.  H.  Bailey,  K.  B.  Barto.  and  E.  Esker.  RADC-TR-85-268 .  Phase  I 
Report,  Rome  Air  Development  Center,  Grlffiss  AFB,  New  York. 

February  1986. 


7.  Transient  Failure  -  Component  degradation  in  the  system  causes  a 
failure  of  a  transient  nature,  resulting  in  a  report  of  a  mal¬ 
function  of  the  system;  the  transient  behavior  subsequently  is 
not  exhibited  during  testing  by  maintenance  personnel. 

8.  Environmentally  Induced  Functional  Error  -  Envlroniaental  condi¬ 
tions.  such  as  vibrations,  pressure,  and  temperature,  cause 
transient  behavior  in  the  systan.  such  that  a  molfunction  is 
reported;  subsequently  maintenance  personnel  cannot  reproduce 
the  conditions  that  caused  the  transient  behavior. 

9.  Flight  Line  Test  Equipment  Error  -  An  error  in  the  test  equipment 
used  at  the  flight  line  identifies  a  good  unit  under  test  as 
being  faulty;  subsequent  maintenance  levels  verify  that  the 
suspect  unit  is  not  faulty. 

10.  Human  Error  at  Flight  Line  -  A  human  error  at  the  flight  line 
results  in  identifying  a  good  unit  under  test  as  fau  Ity^ 
subsequent  maintenance  levels  verify  that  the  suspect  unit  is 
not  faulty. 

11.  Shop  Test  Equipment  Failure  -  An  error  in  the  test  equipment 
used  at  the  shop  level  identifies  a  faulty  unit  under  test  as 
being  good. 

12.  .'luman  Error  at  Shop  Level  -  A  human  error  at  the  shop  level 
results  in  the  identification  of  a  faulty  unit  under  test  as 
being  good. 


Maintenance  personnel  are  unaware  of  the  subclassifications,  and  we 
assume  that  they  follow  normal  system  maintenance  (NSH)  techniques.  Also, 
the  subclasses  with  operator  errors  and  latent  design  error  manifestations 
(both  in  BIT  and  the  host  system)  —  noted  by  asterisks  —  are  false 
alarms  according  to  the  definitions  set  forth  by  Simpson  et  al.^  A  much 
larger  set  of  subclasses,  each  of  tdilch  deals  with  combinations  of  the 
subclasses  across  different  maintenance  levels,  was  considered  to  be  lower 
order  terms  and  was  beyond  the  scope  of  this  effort. 
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3. 4. 1.2  Tt'.e  Taxonomic  Model  of  System  Failure  Events 


In  addition  to  the  factors  driving  CND.  one  would  expect  &  further 
influence  in  the  detection  and  isolation  of  failures  that  is  tied  strictly 
to  the  inherent  testability  of  the  system.  These  factors  may  be  explored 
by  use  of  the  taxonomic  model  for  system  failures  given  in  Figure  3-2. 


The  taxonomic  model  for  system  failures  first  dichotomizes  the  total 
failures  into  the  subclasses  of  detected  and  undetected  failures.  The 
ratio  of  the  numbers  of  detected  failures  to  that  of  total  failures 
constitutes  the  testability  attribute  FFD.  The  detected  failures  are 
further  dichotomized  into  those  that  are  correctly  isolated  and  those  that 
are  not.  The  ratio  of  the  numbers  of  correct  isolations  to  the  total 
number  of  failures  is  the  testability  attribute  FFI. 


There  are  two  types  of  incorrect  isolations.  The  first  is  the 
erroneous  isolation;  the  wrong  uni-  or  component  is  determined  to  be 
faulty.  I  I'n  Reference  1  this  is  termed  a  nonisolation.  The  second  type  of 
incorrect  isolation  results  from  ambiguous  diagnosis.  In  Reference  1. 
this  is  termed  an  improper  isolation.  During  the  fault  isolation  process, 
if  the  procedure  yields  a  class  of  potentially  bad  units,  and  the  test 
procedures  do  not  allow  the  resolutions  to  discriminate  the  faulty  from 
the  good  units  within  the  class,  that  class  is  called  an  ambiguity  group. 

By  necessity  the  entire  class,  both  good  and  bad  units,  must  be  sent  up  to 

subsequent  maintenance  levels.  The  faulty  units  within  the  class  are 

considered  to  be  correct  isolations.  The  good  units  within  that  class 

represent  incorrect  isolations  resulting  from  ambiguous  diagnosis.  In 

Figure  3-2  this  particular  branch  of  the  failure  taxonomy  corresponds  to 

the  "misdiagnosed  failure"  branch  of  the  CND  taxonomic  model  (Figure 

3-1).  This  serves  to  verify  the  pervasive  role  of  the  CND  testability 

attribute  in  the  measurement  of  both  FFI  and  FFD.  I 


L,,. — 


3-9 


3.4.2  CND  Causal  Relationships  that  Affect  all  Testability  Attributes 


The  CND  causal  relationships  are  central  to  all  of  the  testability 
attributes  that  %«e  are  trying  to  predict.  The  functions  derived  here  will 
be  used  in  the  regression  analyses  for  CND  burden.  CND  rate,  isolation 
level,  and  detection  percentage.  From  Figure  3-1,  we  recognize  that  CND 
events  represent  a  composite  of  12  causal  subclasses.  It  is,  therefore, 
possible  to  write  an  equation  for  CND  t.hat  sums  the  members  of  those 
subclasses  as  follows: 


12 

Number  of  CND  events  per  unit  time  -  x. 

i=l  ^ 

where  is  the  number  of  CND  occurrences  per  unit  time  due  to  the  i^" 
causal  subclass  as  shown  in  Figure  3-1. 

Further,  from  the  taxonomic  model  of  Figure  3-1,  we  know  that  the 
false  alarms  represent  specific  subclasses  as  follows: 

«1  *  *2  *  *6- 

We  now  examine  the  functional  relationships  between  the  causal  sub¬ 
classes  of  CND,  represented  by  x^,  and  measurable  design  characteristics. 
These  functional  relationships  will  provide  guidance  as  to  design  data  for 
use  with  regressions  in  the  develojxnent  of  predictive  equations  for  the 
desired  testability  attributes. 

3. 4. 2.1  Operator  Error  (x^) 

Operator  errors  are  functions  of  design  characteristics  that  are 
somewhat  nebulous  and  not  easily  measured.  Those  characteristics  relate 
to  the  operational  complexities  of  the  host  system  that  affect  a  particular 
LRU  and  to  the  level  and  quality  of  support  provided  to  the  system  opera¬ 
tors.  The  latter  of  these  is  a  function  of  ouch  characteristics  as  opera¬ 
tor  experience,  operator  training  background,  average  operator  intelli¬ 
gence,  and  level  and  quality  of  support  documentation.  This  class  of 


design  character:* sties  was  well  outside  the  scope  of  this  project.  Thus, 
the  effect  of  operator  support  was  eliminated  from  consideration. 

Operational  complexity  has  been  well  studied.  Work  in  the  area  of 
sneak  circuit  analysis  suggested  that  design  characteristics  relating  to 
operational  complexity  were  measurable.  Our  Investigations  indicated  that 
operational  c<:xttplexity  would  be  a  function  of  the  following: 

-  The  number  of  controllable  functions  performed  by  LRU 

-  The  number  and  kind  of  LRU-related  operator  controls 

-  The  number  and  kind  of  LRU-related  indicators 

-  Control  labels  related  to  LRU  operation 

-  The  surface  area  of  LRU-related  control  panels 

Of  these,  the  only  measurable  characteristics  (or  surrogate  measures) 
were  the  number  of  LRU  functions,  number  of  external  switches  on  LRU,  and 
number  of  indicators  on  the  LRU.  We  believe  that  these  characteristics 
have  a  compounding  nature  and  vary  with  operational  complexity.  The 
compounding  nature  would  be  represented  by  a  product.  Thus,  the  operator 
errors  term  reduces  to  the  following: 

Xi  =  f^  [(numbers  of  LRU  functions)  *  (numbers  of  external 
switches)  *  (numbers  of  Indicators)] 

3. 4. 2. 2  Latent  BIT  Design  Error  Manifestations  (X2)  and  Latent 
Design  Error  Manifestations  in  LRU  Functions  (xs) 

Latent  design  error  manifestations,  both  in  BIT  and  LRU  functions, 
are  predominately  functions  of  sneak  circuits.  Unfortunately,  sneak 
circuit  analysis  was  outside  the  scope  of  this  effort,  and  neither  the 
sneak  circuit  analysis  nor  the  design  characteristics  required  for  such  an 
analysis  was  acquired.  However,  a  surrogate  measure  of  topological 
complexity  was  derived  based  on  the  concepts  Inherent  in  sneak  circuit 
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analysis.  As  such,  this  synthesized  characteristic,  topological  complexity 
(TC) .  served  as  an  independent  variable  in  the  functional  relationships 
for  X2  and  x^. 

x^  ■  f2  (Topological  Complexity,  BIT) 

Xg  *  fg  (Topological  Complexity) 

Note  that  the  topological  complexity  of  the  BIT  subsystem  is  approxi¬ 
mated  by  that  of  the  LRU.  This  is  because  such  a  measure  for  the  BIT 
subsystem  was  not  attainable.  Also,  note  the  variable  BIT  is  a  1  if  there 

is  a  BIT  subsystem,  otherwise  it  is  0.  Both  x.  and  x.  should  correlate 

z  0 

positively  with  topological  complexity.  This  term,  in  one  form  or  another, 
is  expected  to  influence  each  of  the  testability  attributes. 

Topological  Complexity:  The  objective  of  a  measure  of  topological 
complexity  is  to  quantify  the  likelihood  that  a  latent  design  error  exists 
within  i  system.  Research  in  the  area  of  sneak  circuit  analysis  has 
produced  an  analytic  technique  for  relating  latent  design  errors  to  syst«n 
topologies  and  component  make-up.  Consequently,  sneak  circuit  analysis 
served  as  the  starting  point  for  synthesizing  a  topological  complexity 
characteristic.  Two  approaches  were  pursued: 

-  Measures  based  on  analyses  of  high-level  functional  descriptors 
available  in  Air  Force  Technical  Orders 

-  Measures  based  on  more  in-depth  design  analyses 

Sneak  circuit  analysis  Involves  the  decomposition  of  a  system  into 
five  constituent  topological  patterns  (see  Figure  3-3).  These  topological 
patterns  increase  in  complexity  frcmi  a  "single-line  pattern"  to  an 
"H-pattern."  Each  pattern  has  associated  with  it  a  number  of  sneak  circuit 
analysis  "clues"  for  application  during  the  evaluation  of  a  system.  The 
number  of  clues  for  the  topological  patterns  increases  with  the  complexity 
of  the  pattern.  That  is  to  say,  the  single-line  pattern  has  several 
associated  clues,  whereas  the  H-pattern  has  well  over  100  associated 
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clues.  Bdsed  on  the  numbers  of  associated  clues,  we  reason  that  the 
likelihood  of  finding  a  sneak  circuit  Increases  with  the  complexity  of  the 
topological  pattern.  It  is  reasonable  to  conclude  that  a  linear  combina¬ 
tion  of  the  numbers  of  each  type  of  topological  pattern  found  in  a  system 
could  be  used  to  characterize  the  likelihood  of  sneak  circuits  being 
present  in  that  system. 

If  we  were  to  endeavor  to  analyze  a  system  by  counting  the  various 
topological  patterns,  we  would  be  «»ell  advised  to  complete  the  sneak 
circuit  analysis  and  correct  any  latent  design  errors  found.  The  cost  of 
performing  such  an  analysis  is  high  (prc^ibitively  so  for  this  effort). 

Initially,  we  explored  several  surrogate  measures  based  on  high-level 
functional  block  diagrams.  One  of  these  was  a  measure  of  parallelism  that 
was  shown  to  have  a  high  correlation  with  CND  burden  in  Phase  I.^ 
Similar  measures  were  obtained  by  characterizing  the  number  of  nodes  and 
branches  in  the  functional  block  diagrams  and  normalizing  by  both  the 
number  of  functional  blocks  and  by  the  number  of  shop  replaceable  units 
(SRUs).  In  general,  these  measures  are  subject  to  great  variation  due  to 
differences  in  documentation.  Therefore,  we  endeavored  to  develop  more 
stable  measures. 

To  this  end.  we  observed  that  the  essential  elements  in  the  sneak 
circuit  topological  patterns  are  switches  and  input  and  output  (10) 
signals  (i.e.,  power  feeds  and  grounds).  Puither.  as  shown  in  Table  3-1. 
the  topological  patterns  can  be  characterized  by  the  products  of  their 
input  and  output  signals  and  switches.  This  product  varies  strongly  with 
the  complexity  of  each  pattern,  as  do  the  numbers  of  associated  clues.  It 
stands  to  reason  that  we  might  expect  a  summation  of  these  products  to 
characterize  the  likelihood  of  sneak  circuits  in  a  system. 

Given  the  design  characteristics  chat  were  deemed  to  be  measurable 
for  the  purposes  of  this  study,  the  most  we  were  able  to  accomplish  was  to 
approximate  a  summation  of  the  topological  pattern  products.  Our  surrogate 
measure  characteristic  is  a  product  of  the  number  of  connections  to  each 
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TABLE  3- 

3NEAK  CIRCUIT  TOPOLOGICAL 

1 

PATTERN  COMPLEXITY 

Topological 

Pattern 

10 

Signals 

Switches 

10  X 

Switches 

Single  Line 

2 

1 

2 

Ground  Dome 

3 

3 

9 

Power  Dome 

3 

3 

9 

Combination  Dome 

4 

4 

16 

H-Pattern 

4 

6 

24 

SRU  within  a  given  LRU  and  a  linear  combination  of  the  components  on  each 
SRU  that  could  serve  as  switches.  This  linear  combination  is  necessary  to 
allow  for  the  different  types  of  con^nents  that  may  possess  different 
inherent  switching  capabilities  (e.g..  Integrated  circuits  versus 
transistors) . 


3  4.2.3  Environmentally  Induced  BIT  Errors  (X3)  and  Environmentally 
Induced  Functional  Errors  (xg) 

Environmentally  induced  errors  in  both  the  BIT  subsystem  and  in  the 
fiinctional  LRU  are.  b  definition,  functions  of  environmental  factors. 
Because  the  scope  of  the  study  did  not  allow  detailed  analysis  of  the  BIT 
subsystems,  the  assumption  was  made  that  BIT  is  subject  to  the  same 
environment  as  the  LRU  under  analysis.  Thus,  we  obtain  the  following 
expressions: 


(environment ^»1  fa'  's.  BIT) 
(envi ro.n.:.<jntal  f -tors) 
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The  environmental  factprs  deemed  measurable  In  the  study  were  thermal, 
contamination,  mechanical,  and  electrical.  It  is  expected  that  and 
fg  are  linear  combinations  of  these  factors.  Although  harsh  environ¬ 
ments  are  known  to  have  an  impact  on  failure  rates  of  electronic  equipment, 
the  degree  to  which  the  environment  contributes  to  transient  behavior  and. 
consequently.  CND  events.  Is  not  well  understood.  For  instance,  at  first 
glance,  we  would  expect  a  complex  electronic  system  under  high  environ¬ 
mental  stresses  to  exhibit  high  transient  failure  rates  and  high  CND  rates. 
However,  especially  In  military  systems,  designers  are  extremely  cautious 
when  designing  equipment  for  such  environments.  This  design  philosophy 
should  lead  to  reduced  overall  CND  rates  simply  because  of  the  high  relia¬ 
bility  of  the  systems.  Therefore.  CND  rates  may  actually  vary  Inversely 
with  environmental  factors  and  still  vary  with  CND  as  a  percentage  of 
overall  maintenance  actions. 

3. 4. 2. 4  BIT  Transient  Failures  (X4)  and  LRU  Functional  Transient 
"Failures  (X7 

Transient  failures,  in  general,  'can  result  from  degrading  and  marginal 
components.  It  Is  possible  that  environmental  factors  have  some  type  of 
compounding  effect;  however,  for  every  stable  component  that  environmental 
stress  forces  into  a  marginal  region  of  operation,  there  are  likely  present 
other  marginal  components  that  may  be  stressed  to  hard  failure.  Thus,  we 
expect  marginal  behavior  to  depend  largely  on  the  types  of  components  used 
and  their  failure  mode  characteristics  and  on  the  application  characteris¬ 
tics  of  the  parts. 

For  BIT  transient  failures,  we  can  apply  the  following: 

X4  =  f4  (BIT  component  characteristics) 

Because  the  data  gathering  resolution  did  not  permit  an  evaluation  of 
the  BIT  subsystem  level,  the  component  characteristics  of  the  host  LRU 
were  assumed  to  include  the  BIT  subsystem.  Therefore, 

X4  =  f4  (component  characteristics,  BIT  =1). 


For  LRU  transient  failures. 


x^  >  £^  (component  characteristics). 


The  component  characteristics  deemed  measurable  are  simply  counts  of 
different  types  of  parts  (e.g.,  number  of  Integrated  circuits  ond  number 
of  resistors).  It  is  expected  that  their  effects  on  their  own  transient 
behavior  are.  for  the  most  part,  independent.  The  equations  may  be 
rewritten  as  follows: 


X 


4 


* (number  of  components  of  type  i).  BIT  . 


and 


n 

x_  *  * (number  of  components  of  type  i ) , 

'  1=1  ^ 

where  n  is  the  number  of  distinct  canponent  types,  and  and  are 
relative  weights. 

3. 4. 2. 5  BIT  Hard  Failures  (xg) 

The  hard  failures  that  occur  in  the  BIT  subsystem  are  typically 
predicted  based  on  reliability  characteristics  of  the  subsystem  constituent 
camponents.  Therefore,  a  preliminary  equation  can  be  written: 

Xg  -  fg  (BIT  component  characteristics). 

Because  data  on  BIT  component  characteristics  were  not  available  for 
this  study,  we  made  the  somewhat  tenuous  assumption  that  the  BIT  component 
characteristics  are  similar  to  those  of  the  LRU  it  monitors.  Thus  the 
equation  was  rewritten: 

Xg  =  fg  (LRU  component  characteristics,  BIT). 


In  any  BIT  subsystem,  the  hard  failures  are  functions  of  the  failure 
rates  of  the  BIT'S  constituent  comp^ents.  Component  failure  rates  are. 
to  a  large  degree.  Independent  phe^'mena.  Therefore,  a  linear  combination 
of  component 'type  counts  may  be  used  to  predict  subsystem  failure  rates, 
where  the  coefficients  chosen  by  regression  are  proportional  to  the  failure 
rates  of  the  respective  component  types. 

In  keeping  w^th  our  assumption,  the  BIT  hard  failures  may  be  expressed 
as  a  function  of  the  component -type  counts  of  the  LRU  under  test. 


n 

C.  ‘(number  of  components  of  type  1),  BIT 

1*1 


3. 4. 2. 6  Test  Equipment  Errors  at  Flight  Line  (xg) 

The  performance  characteristics  of  the  test  equipment  used  at  che 
flight  line  can  best  be  estimated  by  achieving  a  thorough  understanding  of 
the  equipment,  Its  design,  and  Its  operational  environment.  None  of  these 
data  were  available  to  this  study.  As  such,  this  term  was  eliminated. 


3. 4. 2. 7  Human  Errors  at  Flight  Line  (x^g) 

Human  errors  at  the  flight  line  are  thought  to  be  a  function  of 
personnel  training  and  support,  test  procedure  complexity,  accessibility, 
and  the  failure  rates  of  the  specific  LRUs.  The  first  of  these,  personnel 
training  and  support,  was  a  characteristic  unavailable  to  this  study.  The 
second,  test  procedure  complexity,  was  measured  in  a  gross  fashion  — 
counting  the  number  of  pages  in  the  Technical  Orders  (TOs).  This  method 
has  an  Inherent  bias  that  Is  not  easily  overcome,  because  different  vendors 
use  different  documentation  styles.  One  possible  solution  to  this  dllenma 
is  to  draw  a  ratio  of  the  flight  line  documentation  page  counts  to  the 
page  count  of  the  Illustrated  parts  breakdown  (IPBs).  This  should  remove 
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both  style  and  LRU  complexity  biases.  The  third  characteristic,  accessi¬ 
bility.  Is  a  function  of  volume,  weight,  the  number  and  types  of  connec¬ 
tors,  and  other  factors  that  tirffect  removability.  In  the  case  of  poor 
accessibility,  the  unit  Is  less  11'  'v  to  be  removed  If  the  evidence  of 
failure  Is  less  than  overwhelming.  .  fourth  characteristic.  LRU  failure 
rate,  may  be  the  most  Important  of  i'  three.  The  more  often  something 
falls,  the  more  likely  It  Is  to  be  ( xaced  whether  It  Is  good  or  bad. 
(An  example  Is  the  multiple  replacement  strategies  comnonly  used  for  such 
Items  as  spark  plugs  In  automobile  maintenance  —  replacing  all  plugs 
rather  than  isolating  u  failed  plug.)  The  equation  may  now  be  written: 


f  /  page  counts  In  TO 
'lO  *  ^10  (page  counts  In  IPB’ 


accessibility,  failure  rate 


There  are  reasons  to  suspect  that  test  complexity  factors  are 
compounded  by  failure  rates  at  extreme  values.  At  moderate  values,  we 
expect  these  two  characteristics  to  exhibit  more  Independent  behavior, 
resulting  In  a  linear  combination  form.  Unlike  failure  rates  and  text 
complexities.  acces&;lblllty  is  expected  to  be  Independent  of  the  other  two 
and  inversely  related  to  CND  events. 


3. 4. 2. 8  Shop  Test  Equipment  Failures  (X|i) 

The  shop  test  equipment  failures  are  functions  of  the  design  and 
construction  of  the  test  equipment  used  In  the  repair  shops.  None  of 
these  characteristics  were  available  for  the  study.  Consequently ^  this 
subclass  was  eliminated. 


3. 4. 2. 9  Human  Errors  at  Shop  Level  (X]^2) 

Human  errors  occurring  at  the  shop  level  are  functions  of  technical 
training  and  support,  shop  level  test  equipment  complexity,  and  LRU 
complexity.  Although  measures  of  LRU  complexity  were  available  to  the 
study,  the  former  two  characteristics  were  not  available.  Further,  it  Is 
likely  that  these  characteristics  form  a  compounding  relationship.  Thus 
It  was  believed  that  data  available  were  Inadequate. 
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3.4.2.10  Suanarv  of  CND  Causal  ttelatlonshlps  to  be  Applied  to 
T#stdbllltv  Attributes 

Of  the  12  ciiusally  oriented  subclasses  of  CND.  three  had  to  be 
eliminated  due  to  unattainability  of  appropriate  design  characteristics. 
In  the  nine  remaining  subclasses,  numerous  design  characteristics  Mere 
eliminated  or  replaced  by  surrogate  measures  or  other  synthesized 
attributes.  In  addition,  numerous  characteristics  were  functionally  used 
for  more  than  one  subclass  (cross  talk).  Although  beginning  with  12 
independent  subclasses  of  CND.  because  of  inadequacies  in  available  data 
the  result  was  a  very  coarse  scale  analytical  model  for  CND.  It  was. 
nevertheless,  adequate  to  guide  the  regression  analysis. 

CND  events  are  a  function  of  the  following: 

-  Operational  complexity  (3. 4. 2.1) 

-  Topological  complexity  (3. 4. 2. 2) 

-  Envirorunental  factors  (3. 4. 2. 3) 

-  Transient  factors  (3. 4. 2. 4) 

-  Comp<jnent  characteristics  (3. 4. 2. 4)  and  (3. 4. 2. 5) 

-  Numbers  of  components  by  type  (3. 4. 2. 4)  and  (3. 4. 2. 5) 

-  Accessibility  (3. 4. 2. 7) 

-  Failure  rates  (3^4.2.?) 

-  Documentation  quality  (3. 4. 2. 7) 

As  noted  in  Phase  I.  the  testability  attributes  FFA.  FFI.  and  FFD  are 
matheoatlcally  related  to  CND  events.  Thus  the  factors  listed  above  will 
influence  all  three  testability  attributes.  Although  specific  guidance  as 
to  anticipated  forms  of  these  parameters  is  given  in  the  cited  paragraphs, 
the  actual  forms  and  significance  were  developed  in  the  regression 
analyses  task  as  discussed  in  Chapter  Five. 
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3.4.3  Additional  Causal  Relationships  Associated  with  FFI 


Figure  3-2  is  the  taxono<nlc  model  used  In  considering  fault  Isolation 
functions.  The  fraction  of  faults  isolated  In  a  system  Is  a  function  of 
that  percentage  of  the  total  of  failure  modes  that  a  given  test  system  is 
configured  to  resolve.  Thus,  all  of  the  failure  modes  and  their  associated 
rates  of  the  rep'  f^able  units  must  be  known.  We  must  also  know  how 
failure  Informat ropagates  throughout  a  system  once  failures  occur. 
Finally,  we  must  understand  vdiat  failure  Information  our  test  system  can 
capture,  along  with  the  reliability  of  that  capture.  Therefore. 

FFI  -  f  (functional  topology,  component  failure  rates,  component 
failure  modes,  test  placements,  test  functions,  test 
reliabilities). 


For  this  study,  all  of  these  design  characteristics  had  to  be  eliminated 
or  replaced  by  coarse  surrogate  measures.  The  resulting  functional 
relationship  will  Include  all  of  the  functions  listed  In  3.4.2.10  as  well 
as: 

FFI  9  f  (topological  measure(s).  functional  complexity,  test 
measure (s).  number  of  components) 


The  topological  measures  used  here  are  different  from  the  topological 
complexity  measures  described  In  previous  subsections.  These  topological 
measures  must  characterize  the  relationships  between  the  functional 
topologies  of  the  uni ts-under- test  and  the  corresponding  test  points. 
Because  of  the  coarseness  of  these  measurable  characteristics,  the  form  of 
this  equation  cannot  be  further  refined.  However,  we  expect  FFI  to  vary 
with  tests  and  inversely  with  components.  Its  behavior  as  a  function  of 
topological  measure (s)  and  functional  complexity  is  not  certain. 

3.4.4  Additional  Causal  Relationships  Associated  with  FFD 

As  is  the  case  with  FFI.  the  percentage  of  faults  detected  will  be 
related  to  such  design  characteristics  as  functional  topology,  failure 
rates,  component  failure  modes,  test  functions  and  placements,  and  test 
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reliabilities.  The  resulting  equation  using  available  characteristics  is 
of  a  form  similar  to  that  of  FPI  and  will  include  all  of  the  functions 
listed  in  3.4.2.10.  as  well  as: 

FPD  s  g  (topological  measureCs) .  functional  complexity,  test 

measure (s).  number  of  components)  , 

This  correspondence  of  FFD  to  FFI  is  logical  given  the  relative  natures  of 
fault  detection  and  fault  isolation.  For  example,  at  the  flight  line 
under  ideal  conditions,  identification  and  r«&oval  of  a  faulty  LRU 
constitutes  a  correct  isolation.  From  the  perspective  of  the  shop,  a 
fault  was  detected  in  the  LRU  pulled.  Similarly,  under  ideal  conditions, 
v4ien  the  shop  technicians  identify  and  remove  a  faulty  SRU,  they  have 
performed  a  correct  isolation.  However,  front  the  perspective  of  the 
depot,  a  fault  has  been  detected  in  that  SRU.  Thus,  we  may  say  that  a 
detection  at  one  maintenance  level  is  equivalent  to  an  isolation  at  a 
lower  level,  and  an  isolation  at  one  maintenance  level  is  a  detection  at  a 
higher  level.  Although  it  is  recognized  that  the  parameters  FFI  and  FFD 
have  the  same  functions,  it  is  the  regression  models  that  should  provide 
the  significance  of  these  functions  to  FFI  and  FFD. 

3.5  CONCLUSION 

The  analysis  in  this  chapter  of  the  form  and  rationale  of  parameter 
ma)ce-up  of  a  testability  prediction  model  for  each  of  CND  burden,  CND 
rate,  FFI.  and  FFD  is  summarized  in  Table  3-2.  Although  certain  forms  and 
tendencies  were  hypothesized,  the  crosstalk  between  numerous  variables 
(such  as  total  components  and  total  components  by  type)  may  cause  compen¬ 
sating  relationships  in  the  final  prediction  equations.  It  is  reccrnimended 
that  trends  be  determined  in  single  variable  correlations.  All  this 
aside,  the  model  provides  a  "common  sense"  quality  control  mechanism  for 
the  regressions  of  data  acquired  during  this  program. 
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TABLE  3-2 


FUNCTIONAL  RELATIONSHIPS  TO  EXAMINE 

k 


Parameter  CND  Rate  a  FAR  CND  Burden  «  ffa  IL  »  ffi  dP  s 


Operational  X 

Complexity 

Topological  X 

Complexity 

Functional 

Complexity 


Environmental  X 

Factors 

Transient  X 

Factors 

Component  v  X 

Characteristics 

Numbers  of  X 

Components 

Accessibility  X 

Failure  Rate  X 

Documentation  X 

Quality 

Topological 

Measures 


X  XX 

X  XX 

X  X 

X  XX 

X  XX 

X  XX 

X  X  X 

X  XX 

*  XX 


Test  Measures  X  X 


*CND  burden  is  not  expected  to  depend  on  failure  rate  because  the  term 
appears  in  both  the  numerator  and  dencxninator . 


CHAPTER  FOUR 


DATA  SOURCES  AND  CHARACTERIZATIONS 


4.1  INTRODUCTION 

Three  categories  of  data  sources  «#ere  identified  In  the  Phase  I 
report:  field  data,  engineering  test- data,  and  design  data.  Although 
field  data  proved  to  be  sufficient  for  estimating  FFA,  the  form  of  the 
Maintenance  Data  Collection  System  (HDCS)  does  not  provide  the  appropriate 
information  for  estimating  FFD  and  FFI.  (As  noted  in  Phase  I.  this  is  not 
a  judgment  of  MDCS,  only  the  recognition  that  MDCS  cannot  be  used  to  esti¬ 
mate  a  quantity  that  it  was  not  designed  to  measure.)  Because  restructur¬ 
ing  of  a  data  reporting  system  was  not  feasible,,  detailed  STAMPS  analyses 
(see  Chapter  Two)  for  24  syst«ns  were  used  in  lieu  of  field  data  for 
estimating  FFD  and  FFI. 

Engineering  test  data  (such  as  maintenance  demonstration  data  and 
operational  evaluation  data)  were  investigated  and  found  not  '>  be  appli¬ 
cable  to  the  needs  of  this  project.  The  primary  difficulty  with  these 
data  is  that  the  normal  evolution  of  system  design  takes  place 
concurrently  with  the  gathering  of  data,  which  leads  to  nonstationary 
effects  in  the  data.  Thus,  the  categories  of  data  sources  to  be  discussed 
in  Phase  II  are  field  data,  STAMP*  analysis  data,  and  design  data. 

4.2  FIELD  DATA 

The  estimator  for  FFA  is  fraction  of  cannot  duplicate  (FCND).  A  false 
alarm  is  defined  as  an  "indication  of  failure  in  the  system  where  none 
exists."  Considering  this  definition,  FFA  is  the  ratio  of  false  alarms  to 
the  sum  of  false  alarms  and  faults  in  the  system.  The  how-malfunctioned 
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(how-mal)  code  for  ChH),  799,  as  reported  in  the  AFTO  349  data  system, 
represents  the  event  that  a  failure  is  reported  (by  BIT,  normal  system 
maintenance,  pilot,  etc.),  and  no  fault  is  found.  A  related  quantity  is 
how-mal  code  812,  tdiich  is  the  event  that  a  failure  is  reported  In  an  LRU 
system  and  fault  Is  found  in  a  different  LRU  system.  Thus,  the  estimator 
for  FFA  is: 


FFA 


number  of  799  how-mal  codes 
number  of  812  how-mal  codes 


23  MA 


^  MA 


wheie 

NA  total  number  of  maintenance  actions. 


A  second  testability  measure  similar  to  FFA  is  the  false-alarm  rate. 
FAR  is  the  number  of  false  alarms  over  a  specified  time  period  divided  by 
that  time  period.  The  estimator  for  FAR  by  LRU  type  is: 


FAR 


_ CND  for  1  year _ 

Operating  hours  for  1  year 


Three  aircraft  were  designated  as  sources  for  field  data  in  Phase  I: 
the  C-5,  F-15,  and  F-16.  For  each  aircraft.  LRUs  were  selected  for  study 
on  the  basis  of  a  reasonable  cross  section  of  aircraft  electronics,  volume 
of  maintenance  activity,  and  availability  of  design  data.  The  LRUs 
selected  for  study  are  listed  in  Table  4-1. 

For  each  LRU,  totals  of  maintenance  actions  and  CND  were  tabulated 
using  the  existing  HDCS.  For  the  C-5,  maintenance  action  and  CND  totals 
were  obtained/  using  the  Malfunction  Analysis  Detection  and  Recording 
System  (NADARS)  as  well  as  using  weekly  summaries  of  maintenance  activity. 
For  the  F-16,  maintenance  actions  and  CNDs  were  totaled  by  LRU  with  a 
tailored  computer  routine  that  uses  two  tapes  generated  under  the  Mainte¬ 
nance  Fault  Listing  Summary  (MFLS)  reporting  system.  For  the  F-15,  total 
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maintenance  action  and  CND  counts  were  obtained  manually  from  AFTO~349 
records.  For  each  aircraft  type,  the  data  collected  span  one  year  of 
maintenance  activity.  A  sunmary  of  the  data  is  listed  in  Table  4-2. 


TABLE  4-2 

SUWfARy  OF  FIELD  DATA 


Aircraft 


Total 


Type  1  Type  2  Type  3 


No.  of  CND  Actions  801  3,472  2,274  6,545 

No.  of  HAS  2.688  10,847  8.985  22,520 


4.3  STAMPS  DATA 
4.3.1  FFI 

A  fault  isolation,  defined  as  identification  by  normal  system  mainte¬ 
nance  (NSN)  of  all  failed  units  within  the  syst«n.  Includes  "ptoper"  fault 
isolation  (only  and  all  of  the  failed  items  are  isolated)  and  “improper" 
fault  isolation  (all  but  not  only  the  failed  items  are  isolated).  Given 
this  definition  of  fault  isolation,  FFI  can  be  defined  as  the  ratio  of 
Isolations  using  NSN  to  faults  within  the  system.  The  Phase  I  effort 
showed  that  we  are  not  currently  measuring  (in  the  AFN-66  data  system) 
parameters  that  can  be  used  to  calculate  this  number. 

STAMP*  calculates  the  similar  measure,  isolation  level.  Under  ideal 
circumstances,  every  system  or  unit  failure  can  be  isolated  to  a  single 
eluent.  Thus,  the  total  possible  number  of  fault  isolations  equals  the 
number  of  elements.  However,  in  actuality,  some  faulty  elements  may  be 
isolated  only  in  ambiguity  groups.  Then  the  number  of  fault  isolation 
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conclusions  will  be  less  than  the  total  possible  fault  isolation  conclu¬ 
sions.  A  fault  isolation  conclusion  is  the  result  of  a  fault  isolation, 
that  is,  a  single  element  that  has  failed,  a  group  of  elonents  that  contain 
the  failure,  or  no  fault  found  (sometimes  referred  to  as  RTOK  in  STANPe 
nomenclature).  IL  measures  this  difference  and  can  be  expressed  as: 


j,  ,  Number  of  fault  isolation  conclusions 
Number  of  elements 

(In  STAMPe,  elements  may  be  functional  or  actual,  as  well  as  system  level 
or  parts  level.) 

With  this  similarity  between  definitions.  IL  was  considered  an  appro¬ 
priate  estimator  for  FFI.  Thus,  IL  meteures  were  collected  from  22  of  the 
available  STAMP*  analyses  as  estimates  of  FFI.  These  systems  are  listed 
in  Table  4-3. 

4.3.2  FFD 

Fault  detection  is  defined  (as  given  by  the  Phase  I  report)  as  an 
indication  by  NSM  that  "the  system  is  not  functioning  properly,  and  this 
Indication  is  the  result  of  a  real  fault  within  the  system."  Building 
upon  this  definition,  FFD  is  the  ratio  of  fault  detection  to  faults  within 
the  system.  (In  the  AFM-66-1  data  system,  we  are  not  currently  measuring 
parameters  that  could  be  used  to  calculate  this  number.) 


The  testability  report  generated  by  STAMP*  includes  the  nondetection 
percentage  measure  that  approximates  the  complement  of  this  quantity. 
STAMP*  considers  each  element  a  potential  fault  or  failure  mode  and  then 
determines  if  that  element  would  be  detected  under  the  current  test 
situation.  If  a  fault  occurs  and  cannot  be  detected,  then  the  system, 
when  retested,  will  appear  "OK."  Thus,  nondetection  of  a  fault  is  ambi¬ 
guous  with  "no  fault  found."  From  this  reasoning: 


NDP  = 


Number  of  elements  not  distinguishable  from  "no  fault  found" 
total  number  of  elements 
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TABLE  4-3 

STAMP*  SYSTEMS  INCLUDED 

IN  STUDY 

AN/ALR-67 

Harpoon 

IP-1276* 

R-2148* 

CARA 

C-10250* 

RFA 

AS-3100* 

BIT 

Special  Receiver* 

OFA 

Computer* 

IFA 

TLI 

AN/ALO-184 

SLP 

RT 

Digital  Section 

CPA 

RF  Section 

RAl 

IQD 

AOC 

APG-63 

TEAMPAC** 

*Addltlonal  derivative  systems 
were  derived  by  deletion  of  10 
percent  and  20  percent  of  tests 
for  additional  data  points. 

**Additlonal  derivative  systems 
were  derived  by  deletion  of  10 
percent  of  tests  for  additional 
data  points. 


To  relate  this  variable  to  PFD.  the  variable  1  -  NDP  =  DP.  or  detec¬ 
tion  percentage,  was  considered.  DP  may  be  represented  as: 

number  of  elements  distinguishable  from  "no  fault  found" 

^  “  total  number  of  elements 


or 


DP  * 


number  of  elements  that  are  detectable 
total  number  of  el^nents 
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Considering  the  similarities  between  the  definitions  for  DP  and  FFD, 
DP  was  deemed  an  adequate  estimator  for  PFD.  Thus.  NDP  was  collected  from 
the  testability  report  for  the  same  22  analyses  composing  the  data  base 
for  FFI.  DP  *  1  -  NDP  was  then  calculated. 

In  compiling  the  data  base  for  FFD.  we  discovered  that  20  out  of  22 
systems  have  a  value  of  1  for  DP.  This  is  not  surprising  because  DP  is  an 
upper  bound  for  FFD  as  discussed  in  Chapter  Two.  Several  alternatives 
were  discussed  to  counter  this  difficulty,  including  presenting  the  equa¬ 
tion  FFD  >  1  as  the  best  predictor  of  FFD.  Leaving  this  alternative  as  a 
last  resort,  another  procedure  was  tried  —  modifying  seven  of  the  20 
STAMPe  analyses  systems  by  randomly  deleting  10  percent  of  the  systems 
tests.  In  addition,  six  of  those  seven  systems  were  modified  by  deleting 
20  percent  of  their  tests  at  random.  With  fewer  tests,  more  elements  \tere 
undetectable  and  the  range  of  values  for  DP  was  greater.  IL  values  were 
also  tabulated  for  the  13  modified  systems  for  comparison  to  the  IL  values 
of  the  unmodified  systems.  It  was  believed  that  if  the  IL  values  from  the 
modified  systems  were  within  predictive  norms  of  the  IL  equation,  then  the 
DP  values  from  the  modified  sysiteros  could  be  included  in  the  data  base  for 
estimating  FFD. 

4.4  DESIGN  DATA 

After  the  "wish  list"  of  design  information  had  been  compiled, 
intermediate-level  and  depot-level  maintenance  manuals  for  several  F-16 
LRUs  were  examined  to  determine  what  design  information  was  available  and 
quantifiable.  From  this  effort,  an  LRU  worksheet  (Appendix  B)  was  devel¬ 
oped  to  aid  in  consistent  and  efficient  data  collection  (see  Chapter 
Three . ) 

This  LRU  worksheet  was  completed  for  every  LRU  from  the  field  data 
group  and  every  LRU  system  from  the  STAMP*  analysis  systems.  Worksheets 
for  field  data  LRUs  were  completed  from  information  in  intermediate-level 
and  depot-level  maintenance  manuals  and  the  illustrated  parts  breakdown 
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and  from  consulting  shop-level  and  fllghtllne  maintenance  personnel. 
Despite  the  simplicity  of  the  worksheet,  some  information  was  difficult  to 
obtain  due  to  variations  In  manuals  written  by  different  vendors.  A 
sunnary  of  trips  taken  to  collect  design  data  for  the  field  data  LRUs  Is 
presented  In  Table  4-4. 


TABLE  4-4 

SUMMARY  OP  DATA  COLLECTION  TRIPS 

Destination  Purpose 

Design  data  for  F-16  LRUs 
Design  and  environmental  data  for  C-5  LRUs 
Completion  of  design  data  for  P-16  LRUs 
Design  and  environmental  data  for  P-15  LRUs 
Envlroiwental  data  for  F-16  LRUs 
Completion  of  design  data  for  C-5  LRUs 


Urlght  Patterson  APB 
Dover  APB 

Urlght  Patterson  APB 
Langley  APB 
MacDlll  APB 
Kelly  APB 


Worksheets  for  STAMP*  analyses  were  completed  using  parts  breakdown, 
blueprints,  and  wiring  diagrams  and  using  Information  provided  by  ARINC 
Research  Corporation  project  engineers  who  coordinated  the  respective 
STAMP*  analyses.  In  addition  to  the  worksheet  data,  numbers  of  tests, 
input  Items,  and  components  used  by  STAMP*  were  included  as  design 
variables. 

4.5  CND  AND  PA  RELATIONSHIP 

A  collection  of  "bad  actor"  data  was  used  in  estimating  the  parameter 
£,  as  described  in  the  modeling  efforts  of  P.hase  I.^  fl  is  an  empirical 
coefficient  that  represents  the  percentage  of  CNDs  that  are  false  alarms. 
A  CND  that  is  a  repeat  CND  on  a  particular  serial  number  LRU  would  be 
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Interpreted  as  a  bad  actor  —  a  failure  whose  Isolation  escaped  normal 
troubleshooting.  If  the  bad  actors  could  be  deleted  from  the  total  CND 
count,  remaining  would  be  the  false  alarms.  It  was  suggested  in  Phase  I 
that  data  from  a  bad  actor  program  be  used  to  empirically  estimate  fi  using 
the  equation: 

FA  ■  CND  -  bad  actors  «  £  CND 

However,  this  approach  was  suggested  under  the  assumption  that  a  bad 
actor  was  a  repeat  CND  unit.  The  actual  bad  actor  data  included  not  only 
repeat  CND  items  but  also  items  with  multiple  occurrences  of  any  .how-mal 
code.  This  made  the  bad  actor  data  impractical  for  use  in  predicting  fi. 
As  a  result.  CND  was  used  as  an  estimator  for  FA  <fi  «  1). 

Despite  this  difficulty,  the  data  for  the  remaining  testability 
attributes  appears  to  be  sufficient  for  deriving  a  prediction  equation, 
with  the  possible  exception  of  FFD. 

4.6  SUMMARY 

Table  4-5  summarizes  the  number  of  observations  (LRUs  or  systems)  for 
analysis  for  each  testability  attribute.  For  each  observation,  the  appro¬ 
priate  estimator  for  the  attribute  and  the  design  data  from  that  system 
have  been  compiled.  These  data  bases  served  as  the  basis  for  the 
regression  analysis. 


TABLE  4-5 

SUMMARY  OF  SYSTEMS  ANALYZED 

Testability 

Number  of 

Attribute 

Estimator 

Systems 

FFA 

CND/HA  =  CND  burden 

38 

FAR 

CND/Op  hours  =  CND  rate 

38 

FFI 

IL 

22 

FFD 

DP 

35 
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CHAPTER  FIVE 


DATA  ANALYSIS 


5.1  INTRODUCTION 

The  analyses  off  the  cffirinators  .(for  the  testability  attributes  FFA. 
FAR.  FFI.  and  FFD  t«ere  directed  to  the- development  of  predlctlcxi  equatlms 
for  CND  burd<?n.  CND  rate.  IL,  and  DP.  For  these  analyses,  computer  soft> 
ware  was  selected  and  reijression  analysis  tMS  performed.  The  discussion 
of  Che  following  sections  provides  the  theoretical  and  math«natlcal  Justi¬ 
fy  flcatlon  for  the  final  form  of  the  prediction  equations.  The  actual 
computation  euid  use  of  these  equations  are  discussed  In  Chapter  Six. 

5.2  SELECTION  OF  SOFTWARE 

Software  was  selected  from  a  wide  variety  of  computer  packages  based 
upon  the  following  features: 

Basic  statistical  measures 

Plotting  capability  (histograms,  scatter  plots) 

Multiple  linear  regression 
Nonlinear  regression 
Adequate  storage  capacity 

These  features  and  the  capability  criteria  led  to  the  choice  of  two 
packages  for  use  in  this  work:  a  linear  regression  package.  "The 
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Statistician"  by  Quant  Systens.  and  a  nonlinear  regression  package, 

13 

“Harqfit"  by  Schreiner,  et  al..  Mhlch  uses  the  Marquardt  algorithm. 
Results  from  a  simple  regression  with  The  Statistician  were  compared  with 
ttio  other  linear  regresslw)  packages  to  verify  the  accuracy  of  the  routine. 
ARINC  Research  developed  microcomputer  packages  were  occasionally  used  for 
trending  analysis  and  small  computation  routines. 

5.3  GENERAL  NOTES  ON  THE  ANALYSIS 
5.3.1  Use  of  Stepwise  Regression 

The  regression  package  used  to  analyse  the  data  Included  a  stepwise 
regression  routine.  This  routine  selects  the  variable  (from  a  user- 
selected  list  of  Independent  variables)  that  yields  the  highest  contribu¬ 
tion  to  the  explanati<xi  of  the  variance  of  the  dependent  variable.  The 
routine  then  computes  partial  correlations  and  selects  the  variable  from 
the  remaining  list  that  has  the  highest  contribution  given  the  presence  of 
the  first  variable.  Continuing  in  this  manner,  the  routine  systematically 
adds  one  variable  at  a  time  to  the  regressi<»i  until  the  list  is  depleted. 
This  routine  proved  useful  in  determining  the  interaction  of  dlfferqpt 
variables. 


There  are  a  few  considerations  in  selecting  the  "best"  equation  for 
each  of  the  estimates  for  the  testability  attributes:  what  constitutes 
"best"  and  how  to  deal  with  illogical  variables.  Illogical  variables 
include  variables  that  do  not  behave  as  expected. 


^^The  Statistician.  Quant  Systems.  Charleston.  SC.  1983. 

Schreiner,  et  al.  "Nonlinear  Least  Squares  Fitting."  PC  Tech 
Journal,  Vol.  3.  No.  5,  May  1985.  pp.  170-190. 
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5. 3. 2.1  What  Constitutes  "B»st" 


To  be  certain  that  the  best  possible  equation  has  been  chosen,  every 
possible  conblnatlOT  of  variables  should  be  tested.  This  Is  a  combina¬ 
torial  problem.  The  number  of  regressions  necessary  to  do  every  combina¬ 
tion  Is  given  by: 


where  N  is  the  number  of  independent  variables. 


If  there  were  10  independent  variables,  this  would  Indicate  1.024  regres¬ 
sions  to  compute  and  compare.  However,  the  number  of  variables  for  CND 
burden  alone  Is  closer  to  80.  An  exhaustive  search  of  every  possible 
combination  of  variables  was  beyond  the  scope  of  this  effort.  Thus,  our 
recommended  equations  represent  the  best  equation  that  we  have  computed, 
guided  by  the  results  of  the  analytical  modeling  (the  analytical  model  Is 
discussed  In  detail  In  Chapter  Three.)  The  stepwise  linear ’regression 
routine  discussed  In  5.3.1  also  provided  some  order  to  the  significance  of 
the  data  and  reduced  the  necessary  combinations  to  be  explored. 


5. 3. 2. 2  Illogical  Variables 

The  areas  In  which  a  variable  exhibited  unexpected  behavior  varied. 
These  areas  and  the  procedures  used  to  deal  with  the  problems  are  outlined 
below. 


On  occasion,  a  variable  that  was  not  expected  to  be  a  major 
factor  would  appear  to  be  significant  in  an  equation.  Often 
these  variables  were  Incl'jded  for  control  to  help  our  own  under¬ 
standing  of  the  problem  and  to  verify  our  analytical  work.  In 
such  cases,  the  analytical  models  were  review^  to  suggest  a 
possible  reason  for  the  significance.  One  such  variable  was 
WEIGHT,  which  Initially  held  substantial  significance  In  deter¬ 
mining  CND  burden.  The  analytical  model  suggested  that  weight 
might  factor  into  the  accessibility  of  an  LRU.  along  with  volume 
and  the  number  of  external  connectors.  When  weight,  volume,  and 
external  connectors  were  included  simultaneously  in  a  regression, 
tfeight  became  insignificant,  confirming  the  analytic  model. 
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As  part  of  the  analytical  modeling,  the  expected  sign  of  the 
Individual  correlation  between  each  variable  and  the  appropriate 
dependent  variable  was  postulated.  At  tiroes,  the  postulated  sign 
and  the  actual  sign  differed.  These  instances  raised  a  flag  to 
Investigate  the  analytical  model  for  either  an  explanation  or  a 
revision  of  the  Independent  variables.  For  Instance,  several  of 
the  early  topological  variables  had  a  negative  correlation  with 
CND  burden  and  CND  rate.  Indicating  that  the  more  functionally 
complex  systems  had  fewer  CND  actions  and  vice  versa.  Review  of 
the  analytical  model  revealed  better  measures  of  topological 
complexity  which  correlated  positively  with  CND  burden  and  CND 
rate. 

Note  that  the  concern  over  an  unanticipated  sign  is  for  the 
Individual  correlation  as  well  as  the  coefllclent  of  the  variable 
in  an  equation.  The  latter  could  reflect  correlation  between 
Independent  variables,  and  the  unexpected  sign  on  one  variable 
may  indicate  compensation  for  the  over  contribution  of  another 
variable. 

A  few  variables  were  used  as  control  variables.  These  represented 
scMne  difference  or  expected  difference  that  held  little  value  as 
a  design  characteristic.  If  the  control  variable  was  significant, 
it  was  compared  against  other  variables  until  the  information 
represented  by  the  control  variable  was  covered  by  a  combination 
of  appropriate  design  characteristics.  One  such  variable  was 
PLANE,  a  categorical  variable  that  indicates  the  aircraft  type 
(C-5,  F-15.  F-16)  that  the  LRU  resided  on.  Although  initially 
significant,  the  information  represented  by  PLANE  was  eventually 
better  represented  by  other  variables  (in  particular,  the 
environmental  variables)  that  did  not  indicate  aircraft  type. 

5.3.3  Predictor  Development  Hethcdoloqy 

In  the  development  of  prediction  equations,  a  broad  spectrum  stepwise 
linear  regression  was  performed  to  find  the  most  significant  measures  in 
accounting  for  variation  in  the  dependent  variable  (CND  burden,  CND  rate, 
IL,  and  DP.)  These  high-value  measures  were  then  compared  to  the  anal¬ 
ytical  workup  in  Chaptei  Three  and  variables  that  best  represented  the 
derived  functionalities  were  developed  through  subregressions  of  the 
dependent  variable.  These  subregressions  were  done  both  by  stepwise  and 
nonlinear  combinations,  and  output  was  again  compared  to  derived  function¬ 
alities.  Finally .  the  synthesized  variables  were  then  used  in  a  stepwise 
linear  regression.  As  can  be  seen,  these  last  few  steps  required  several 
iterations  and  will  account  for  the  differing  forms  of  a  term  like 
"topological  complexity"  as  it  is  fine  tuned  to  the  appropriate  dependent 


variable.  In  all.  we  performed  more  than  300  regression  runs  to  obtain 
the  equation  sets  presented  In  this  chapter. 


5.3.4  Procedure  for  Validation 

The  prediction  equations  were  validated  by  application  to  one  or  more 
data  sets  not  used  in  the  development  of  the  equations.  Thus,  twc  systems 
were  deleted  from  the  total  data  set  for  CND  burden  and  CND  rate,  and  one 
system  was  deleted  for  IL  and  DP.  After  being  used  to  verify  the  equa¬ 
tions,  these  LRUs  were  then  folded  back  into  the  data  bases  to  enhance  the 
flriul  form  of  the  equation. 

5.3.5  An  Explanation  of  Some  Regression  Statistics  and  Tests 

Various  statistics  %eere  used  to  compare  regression  equations  and 
determine  the  significance  of  the  equation  and  variables.  These  statis¬ 
tics  are  as  follows: 


r2  -  A  normalized  compl«nent  of  the  square  difference  between 
the  observed  values  and  the  equatlonal  values,  ranging  from  0  to 
1  inclusive.  As  such,  It  Indicates  the  percentage  of  the  variance 
in  the  dependent  variable  that  is  explained  by  the  equation  (0 
represents  no  explanation  and  1  represents  perfect  explanation). 
r2  Is  used  to  compare  regression  equations  and,  thus,  can  be 
used  as  a  measure  of  "goodness  of  fit."  This  sometimes  is  called 
the  correlation  factor. 

F  -  Measures  the  global  utility  of  the  predictive  equation.  The 
F  statistic  tests  that  one  or  more  of  the  coefficients  are  nonzero 
by  comparison  with  a  tabulated  F  value.  The  tabulated  F  value  is 
based  on  the  number  of  variables  and  the  number  of  observations 
for  a  given  significance  level.  The  observed  F  value  should  be 
greater  than  the  tabulated  value  for  the  utility  of  the 
equation. 

t  statistics  -  Indicate  the  degree  to  which  each  variable  is 
significant  in  the  equation.  The  t  statistic  for  each  variable 
tests  that  its  coefficient  is  nonzero  by  comparison  to  a  tabu¬ 
lated  value.  The  tabulated  value  Is  based  on  the  number  of 
observations  and  a  given  significance  level .  The  absolute  value 
of  the  observed  t  statistic  should  be  greater  than  the  tabulated 
value  for  the  inclusion  of  that  variable  in  the  equation. 


T.  McClave  and  P.  G.  Benson,  Statistics  for  Business  and  Economics. 
Dellen  Publishing  Company,  San  Francisco.  California,  1982.  pp.  472-478. 


5.4  ANALYSIS  OF  CND  BURDEN  AND  CND  RATE 
5.4.1  Variables 

For  each  analysis,  variables  fell  into  three  classifications:  initial 
design  variables,  combinations  of  initial  variables,  and  variables  synthe¬ 
sized  by  regression.  Initial  design  variables  consist  primarily  of  counts 
and  measures  obtained  from  the  worksheets  (see  Appendix  B).  In  addition, 
other  design  variables  include  parts  counts  normalized  by  number  of  SRUs, 
a  yes  or  no  BIT  variable,  and  a  percent  digital  variable.  The  environ¬ 
mental  variables  —  thermal,  contaminants,  mechanical,  and  electrical  — 
%fere  calculated  by  summing  the  number  of  sensitivities  ("yes"  responses) 
within  each  category-  A  more  complete  list  of  the  initial  design  vari¬ 
ables  is  contained  in  Appendix  C. 

The  second  category  of  design  variables  Includes  those  variables 
created  from  combinations  of  initial  variables.  Host  combinations 
suggested  themselves,  such  as  volume  and  the  densities.  Other  combina¬ 
tions  were  developed  when  consider  I;-:;  the  physical  situations  that  might 
cause  CNDs,  such  as  the  Interactions  of  power  and  the  environment  with 
other  variables.  Also  in  this  category  is  the  number  of  failures  and 
failure  rate.  Appendix  C  contains  a  listing  of  combinations  of  initial 
variables. 

Finally,  a  nuiober  of  variables  were  synthesized  by  regression.  These 
are  variables  that  were  believed  to  combine  additively  rather  than  multl- 
piicatl./oly.  Multiple  regression  with  the  variables  to  be  combined  as  the 
Independent  variables  and  CND  burden  or  CND  rate  as  the  dependent  variable 
yielded  the  relative  proportion  of  each  variable  in  predicting  the  depen¬ 
dent  variable.  Using  these  proportions  as  weights,  a  new  variable  was 
created  by  the  sum  of  these  weighted  variables.  Included  in  than  category 
are  the  various  parts  complexity  measures.  Appendix  C  presents  variables 
synthesized  by  regression. 


5.4.2  CND  Burden.  Estimator  for  FFA 


5. 4. 2.1  The  Prediction  Equation 

The  equation  developed  to  predict  CND  as  a  percentage  of  maintenance 
actions  Is  as  follows: 


0 

CND/MR  -  •  Fctn 
100 


IF  Fctn<0 
IF  OSFctnilOO 
IF  100<Fctn 


where: 

Fctn  *  25.8  4*TOP4  +  0.003*THRMCON  -  0.076*THRMCMPLX  -  0.002*ACCESS. 


The  variables  Included  In  this  equation  cover  a  wide  range  of  possible 
causes  of  CNDs.  The  following  descriptions  of  each  va  '.able  suggests  Its 
Impact  on  CND  burden. 


TOP4  -  The  varlabl  ?  TOP4  Is  a  measure  of  the  topological  complex¬ 
ity  of  the  LKU.  computed  from  the  functional  block  diagram,  It 
measures  t.  ^  number  of  parallel  functional  paths  relative  to  the 
number  of  Js.  This  variable  Is  Identical  to  the  parallelism 
variable  frum  Phase  I.^  This  variable  encompasses  characteris¬ 
tics  that  effect  CND  causal  subclasses  described  in  subsections 
3. 4. 2.1  and  3. 4. 2. 2. 

THRMCON  -  Intended  to  measure  the  Interaction  between  the  number 
of  interconnects  and  a  thermally  sensitive  environment,  this 
variable  indicates  an  Internal  connection  that,  yAien  heated,  may 
or  may  not  conduct  as  expected,  either  conducting  a  signal  Incor¬ 
rectly  or  failing  to  conduct  the  signal  at  all.  This  encemipasses 
factors  discussed  in  subsections  3. 4. 2. 3  and  3. 4. 2. 4. 

THRMCMPLX  -  This  variable  reflects  the  interaction  between  the 
thermal  environment  and  thermally  sensitive  components.  As  the 
LRU  is  exposed  to  external  thermal  influences,  these  components 
may  "chatter,"  adding  noise  to  the  signal.  This  distorted  signal 
may  cause  a  failure  indication  that,  when  the  LRU  is  removed  from 
the  thermal  stress,  cannot  be  duplicated.  ''Although  this  variable 
has  a  negative  coefficient,  the  single  variable  correlation  with 
CND  burden  Is  positive,  as  expected.)  This  variable  also  reflects 
subsections  3. 4. 2. 3  and  3. 4. 2. 4. 
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ACCESS:  -  The  variable  ACCESS  Indicates  the  influence  of  an  LRU's 
accessibility  on  the  reported  number  of  CKD  events.  A  technician 
will  be  more  certain  that  an  LRU  is  in  need  of  troubleshooting 
before  removing  it  from  the  aircraft  if  the  LRU  is  heavy  and  has 
many  connectors.  Subsections  3. 4. 2. 7  and  3. 4. 2. 4  relate  to  the 
significance  of  this  variable. 

Note  that  THRNCON,  THRHCMPLX.  and  ACCESS  also  exhibit  crosstalk  with  the 
concepts  of  subsection  3. 4. 2. 5.  A  summary  of  the  regression  analysis  for 
CNO  burden  is  presented  in  Table  5-1. 

5. 4. 2. 2  Validation  of  Equation  for  CND  Burden 

Two  systems  were  withheld  frcmi  the  regression  analysis  for  CND  burden 
to  be  used  as  verifiers  of  the  predictive  ability  of  the  equation.  For 
system  1.  the  equation  predicted  a  CND  per  maintenance  action  value  of 
21.7  percent,  compared  to  the  field  value  of  34.8  percent.  The  equation 
predicted  a  value  of  19.6  percent  for  systeir<  2.  which  had  an  observed  CND 
burden  of  23.4  percent.  These  results  and  the  corresponding  prediction 
Intervals  are  presented  in  Table  5-2.  Figure  5-i  presents  the  observed 
versus  predicted  values  together  with  the  two  verification  LRUs.  The 
verification  was  considered  adequate. 

5. 4. 2. 3  Final  Equation  for  CND  Burden 

.lEter  the  predictive  ability  of  the  equation  was  verified,  the  two 
validation  systems  were  folded  back  into  the  data  set  to  compute  a  final 
form  of  the  equation  using  all  the  data  at  our  disposal.  Note  that  the 
form  or  the  equation  and  the  choice  of  independent  variables  remain 
fixed.  Inclusion  of  the  additional  tw  systems  is  only  to  nake  minor 
adjustments  in  the  conotanr  and  coefficients.  The  resulting  equation  is: 

fO  IF  Fctn<0 

Fctn  IF  0<Fctn<100 

^  100  IF  :00<Fctn 

where: 

Fctn  •-  4-  3.9*'10P4  +  0 .003*THRMCON  -  0.077*TiJRMCMPLX 

■- J02*ACCESS. 


TABLE  5-1 

EQUATION  AND  STATISTICS  FOR  CND  BURDEN 


a.  EQUATION  SUmUtY 


Variable 

Coefficient 

Std.  Error 

T-Value 

Coaiputatlon 

CONSTANT 

25.84r 

3.516443 

7.348818 

TOP4 

4:075859 

1.058394 

3.850984 

(cross  count )/No.  of  SRUs 

THRMCON 

3.429899E-03 

5.444746E-04 

6.299465 

(thenulti)  No.  of  Inter¬ 
connects 

THRHCNPLX 

-7.606546E-02 

2.18S168E-02 

-3.48099 

(thermal't^l )  *  (ICS'«-10*lnd-2 . 4* 
cap)/No.  of  SRUs 

ACCESS 

“2.006753E-03 

4.82S38E-04 

-4.158764 

voluM'»S00*connectors 

b.  ANALYSIS  OF  VARIANCE 


Sources  of  Variation 

SS  OF  NSE 

Regression 

3064.358  4  766.0894 

Error 

1773.955  31  57.22436 

Total  SS 

4838.313  35 

F  -  13.38747 

R  Squared  0.6333525 

c.  F  TEST  STATISTICS 

Regression  F  -  13.39 

ft  • 

0.05  0.025  0.01  0.005 

*'4.31  • 

2.68  3.23  4.00  4.59 

99.5%<Signiflcance  Level 

d.  t 

TESTS  STATISTICS 

Variable 

Regression 

ABS  (t  Value) 

TOP4 

3.85 

THi<MCON 

6.30 

THRMCMPLX 

3.48 

ACCESS 

4.16 

to  /2,30  “  2.15:  o  *0.01 


Minimum  Significance  Level  for  all 
Variables  -  99% 


TABLE  5-2 


VALIDATION  OF  EQUATION  FOR  CND  BURDEN 


Variable 

Systen  1 

System  2 

TOP4 

1.46 

0.56 

T!!RMC0N 

1430.0 

2274.0 

THRMCNPU 

-21.4 

-22.64 

ACCESS 

8268.0 

9013.0 

PREDICTED  VALUE 

21. 7\ 

19.6% 

ACTUAL  VALUE 

34.8% 

23.4% 

95\  PREDICTION  INTERVAL 

(6.48.  37.0) 

(4.11.  35.0) 

Table  5-3  summarizes  the  regression  analysis  for  this  final  equation  for 
CND  burden.  Included  in  this  analysis  is  a  scatter  plot  of  predicted  CND 
burden  values  versus  the  actual  field  values  (Figure  5-2). 


For  ease  in  using  this  equation  to  obtain  a  predicted  value,  a  work¬ 
sheet  indicating  the  necessary  measures  and  calculations  is  found  in 
Chapter  Six. 

5.4.3  CND  Rate,  Estimator  for  FAR 
5. 4. 3.1  The  Prediction  Equation 

The  equation  developed  to  predict  CND  per  operating  hours  of  the  LRU 
is  as  follows: 


-  {  Fctn 

where: 


IF  Fctn<0 
IF  OSFctn 


Fctn  =  -0.0029  +  0.381*FLRRATE  *■  2.7*10~5*TRANSIENT 
+  5.7*10-il*TC7. 


Actual  CND  Burden  (%  of  Malntenanc 


Predicted  CND  Burden  (\  of  Maintenance  Actions) 


FIGURE  5-1 

SCATTER  PLOT  OF  OBSERVED  VERSUS  HIEDICTED  VALUES  FOR 
CND  BURDEN  -  INITIAL  EQUATION 


TABLE  i5-3 


FINAL  EQUATTCii  FOR  CM)  BURDEN 


a.  EQUATION  SUMWnr 

Variable 

Coefficient 

Std.  Error 

T-Valu» 

Coaputatlon 

COieTANT 

25.93132 

3.563411 

7.277107 

10P4 

3.918272 

1.059272 

3.699024 

(CLocs  countl/No.  of  SRUs 

THRNCON 

3.295488E-03 

5.4488058-04 

6.048092 

(thetaaWD'No.  of  Inter- 

conneccs 

THRHCNPU 

-7.705728E-02 

2.2162S9E-02 

-3.476908 

(VhecMUl  )*ICs4^10*lnd-2.4* 

cap)/No.  of  SRUs 

ACCESS 

-1.861481E-03 

4.788547E-04 

-3.88736 

volUMe50U*connectors 

b.  AMIU.¥SIS  or  VMtXAICE 


Sources  of  variation 

SS 

DF  HSE 

Regression 

Error 

2963.313 

1943.873 

4  740.8281 

33  58.90525 

Total  SS 

F  •  12.57661 

R  Squared  0.6038721 

4907.186 

37 

c.  F 

TEST  STATISTICS 

Regression  F  •  12.58 

a  • 

^4.33  ■ 

0.05  0.025 

2.66  3.20 

0.01  0.005 

3.96  4.52 

99.S%<signltlcenc«  Level 


d.  C  TESTS  STATISTICS 


Regression 

Variable  (t  value) 


TOP4  3.70 
THRNCON  6.05 
THRNCHPLX  3.48 
ACCESS  3.89 


to  /2,30  -  2.75;  o  -  0.01 

Hlnlmun  Significance  Level  for  all 
Variables  -  99% 
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Actual  CND  Burden  (%  of  Halntenance  Actions) 


Predicted  CND  Burden  (%  of  Maintenance  Actions) 
FIGURE  5-2 

SCATTER  PLOT  OF  OBSERVED  VERSUS  PREDICTED  VALUES  FOR 
CND  BURDEN  -  FINAL  EQUATION 


ine  units  are  CND  events  per  operating  hour.  The  variables  reflect 
different  aspects  of  tlie  causes  of  CNDs.  The  following  description  of 
each  variable  suggests  Its  Inpact  on  CND  rate. 


FLRRATE  -  FLRRATE  represents  the  actual  failure  rate  of  the  LRU. 
As  an  item  Is  known  to  fall  acre  often,  that  item  Is  renoved  from 
the  aircraft  more  often  for  any  failure  Indication.  The  more 
often  the  LRU  Is  removed,  the  more  chance  it  has  of  not  being  the 
failed  item  for  that  maintenance  action.  Thus,  the  CND  rate  will 
rise  with  tb  failure  rate.  This  variable  relates  to  the  anal¬ 
ysis  in  subs.  4ion  3. 4. 2. 7. 

TRANSIENT  -  The  variable  TRANSIENT  Indicates  those  components 
that  «iere  significant  In  ctmtrlbutlng  to  transient  or  Intermittent 
behavior.  For  the  data  set  we  collected,  these  significant 
components  Include  relays,  capacitors.  Integrated  circuits  (ICs), 
resistors,  and  transistors.  This  variable  represents  crosstalk 
between  terns  in  subsections  3. 4. 2. 4  and  3. 4. 2. 5. 

TC7  -  Designed  to  reflect  the  topological  complexity  concept  of 
sneak  circuit  analysis.  TC7  Is  a  product  combination  of  intercon¬ 
nects  (which  are  the  Input  and  output  of  the  topological  patterns) 
and  the  various  components  that  may  act  as  switching  devices. 
For  the  data  set  we  collected,  the  switching  components  »*hlch 
were  significant  Include  ICs.  transistors,  relays,  and  internal 
switches.  Subsections  3. 4. 2. 2.  3. 4. 2. 4.  and  3. 4. 2. 5  Indicate  the 
Impact  of  this  variable. 


A  sunmary  of  the  regression  euialysls  for  OID  rate  is  presented  in 
Table  5-4. 


5. 4. 3. 2  Validation  of  Equation  for  CND  Rate 

As  done  for  CND  burden,  two  systems  were  withheld  from  the  regres¬ 
sion  analysis  to  be  used  as  veiiflers  of  the  predictive  ability  of  the 
equation.  For  system  1,  the  equation  predicted  a  CND  rate  of  0  per  1,000 
operating  hours,  compared  to  the  field  value  of  0.99  CND  maintenance 
actions  per  1,000  operating  hours.  For  system  2,  which  had  an  observed 
CND  rate  of  4.9  CND  actions  per  1,000  operating  hours,  the  equation 
predicted  a  CND  rate  of  5.3  CND  actions  per  1,000  operating  hours.  These 
results  and  the  corresponding  prediction  intervals  are  sunmarlzed 
graphically  in  Figure  5-3,  and  data  are  listed  In  Table  5-5.  The 
verification  was  considered  excellent. 
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TABLE  5-4 

EQUATION  AM)  STATISTICS  FOR  CM)  RATE 

a.  EQUATION  SUMARY 

Variable 

Coefficient 

Std.  Error 

T-Value 

Coaputatlon 

CONSTANT 

-2.953223E-03 

4.994529E-04 

-5.912917 

FLRPXTE 

0.3807576 

2.042714E-02 

18.63979 

(NA-CM))/LRU  operating  hr/yr 

TRANSIENT 

2.668999E-05 

4.166393E-06 

6.40602 

( iCsM  1  *  relays'»'2*  cap 
♦2*res-9*xstrs)/No.  SRUs 

TC7 

5.691348B-11 

2.589693E-11 

2.197523 

lntcon*(lCse30*nt. ' 
-160*relays-960*»M) 

b.  AMU.y3IS  OP  VMtIMCS 


Sources  of  variation 

SS 

OF 

NSE 

Regression 

2.402053E-04 

3 

8.006845E-05 

Error 

2.134589E-05 

32 

6.670589B-07 

Total  SS 

F  -  120.032 

R  Squared  0.0183874 

2.615512E-04 

35 

C.  F  TEST  STATISTICS 


Regression  F  >  120.03 

tt  -  0.05  0.025  0.01  0.005 

F3,32  •  2.91  3.56  4.46  5.19 

99.5%<Slgnlflcancc  Level 


d.  t  TESTS  STATISTICS 


Regression 

Variable  ABS  (t  value) 


FLRRATE  18.6 

TRANSIENT  6.41 

TC7  2.20 

^a/2.30  “  2.04;  a  ■  0.05 

Minimum  Significance  Level  for  all 
Variables  -  95% 


observed  C 
(Nunber  of  CM)  Events  per 


Predicted  CND  Rate 

(Number  of  CND  Events  per  1000  Operating  Hours) 


FIGURE  5-3 

SCATTER  PLOT  OF  OBSERVED  VERSUS  PREDICTED  VALUES  FOR 
CND  RATE  -  INITIAL  EQUATION 


TABLE  5-5 

VALIDATION  OF  EQUATION  FOR  Ot>  RATE 
(values  for  failure  rate  and  CND  rate  are 
per  operating  hour) 


Variable 

System  1 

System  2 

FLRRA7E 

0.0C186 

0.01606 

TRANSIENT 

70.4 

113.8 

TC7 

147.862 

-15.370.875 

PREDICTED  VALUE 

0 

0.0053 

ACTUAL  VALUE 

0.00099 

0.0049 

95%  PREDICTION  INTERVAL 

(0.  0.0013) 

(0.0035.  0.0072) 

5. 4. 3. 3  Final  Equation  for  CND  Rate 

After  the  oredlctlve  ability  of  the  equation  «ias  verified,  the  two 
validation  systems  were  folded  back  Into  the  data  set  to  compute  a  final 
form  of  the  equation  using  all  the  data  at  cHir  disposal.  Note  that  the 
form  of  the  equation  and  the  choice  of  independent  variables  remain  fixed. 
Inclusion  of  the  additional  two  systems  is  only  to  make  minor  adjustments 
in  the  constant  and  coefficients.  The  resulting  equation  is: 

{0  IF  Fctn<0 

."■ctn  IF  OSFctn 

where: 

Fctn  =  -0.0028  +  0.375*FLRRATE  +  2.6-10'5*TRANSIENT 
+  5.9*10"-^i*TC7 

Table  5-6  sunnarizes  the  regression  analysis  for  this  final  equation  for 
CND  rate.  Included  in  this  analysis  is  a  scatter  plot  of  predicted  CND 
rate  values  versus  the  actual  field  values  in  Figure  5-4. 


s 


TABLE  5-6 

FINAL  EgUATION  FOR  CND  RATE 

a.  EQUATION  SUMMRY 

Variable 

Coefficient 

Std.  Error  T-Value 

Cooputatlon 

CONSTANT 

FLRPATE 

TRANSIENT 

TC7 

-2.780678E-03 

0.3749341 

2.563411E-05 

5.87789E-11 

4.869777E-04  -5.710073 
2.Q04064E-02  18.70869 
4.097731E-06  6.255683 

2.402032E-11  2.447049 

(MA-CND)/LRU  operating  hr/yr 
( ICs+41 * relays+2*cap+2* res 
-9*xstrs)/No.  SRU 

Intcon*  (ICS‘*-30*xstrs 
-160*relays-960*sw) 

b.  ANALYSIS  OF  VARIANCE 


Sources  of  Variation 

SS 

DF 

HSE 

Regression 

2.477943E-04 

3 

8.2598ilE-05 

Error 

2.319i24E-05 

34 

6.820954E-07 

Toral  SS 

F  121.0947 

R  Squared  0.9144189 

2.709856E-04 

37 

C.  F  TEST  STATISTICS 


Regression  F  >■  i21.09 

a  -  0.05  0.025  0.01  0.005 

F3,34  -  2.88  3.53  4.41  5.1 

99,5%<Slgnlflcance  Level 


d.  t  TESTS  STATISTICS 


Variable 


Regression 
ABS  (t  value) 


FLRRATE 

TRANSIENT 

TC7 


18.7 

6.26 

2.45 


ta/2,30  ■  2.04:  a  -  0.05 

Mlnimun  Significance  Level  for  all 
Variables  -  95\ 
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1 


xmS 


Actual  CND  Rate 

(Number  of  CND  Events  per  1000  Operating  Hours 


0  2.0  4.0  6.0  8.0  10.0 


Predicted  CND  Rate 

(Number  of  CND  Events  per  1000  Operating  Hours) 
FIGURE  5-4 

SCATTER  PLOT  OF  OBSERVED  VERSUS  PREDICTED  VALUES  OF 
CND  RATE  -  FINAL  EQUATION 
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For  ease  in  using  this  equation  to  obtain  a  predicted  value,  a  work¬ 
sheet  indicating  the  necessary  measures  and  calculations  is  found  in 
Chapter  Six. 


5 . 4 . 3 . 4  Remarks  on  the  Differences  in  the  CKJD  Equations 

Before  beginning  any  regression  analysis,  it  was  decided  to  allow 
statistical  significance  to  guide  the  form  of  the  prediction  equations. 
Rather  than  forcing  certain  independent  variables  in  the  equations,  vari¬ 
able  selection  was  determined  through  the  regressions  th«aselves.  This 
policy  resulted  in  CND  burden  and  CND  rate  equations  with  dissimilar 
terms.  The  position  of  the  failure  rate  term  is  the  primary  contributor 
to  this  difference.  In  the  case  of  CND  burden,  the  failure  rate  term  is 
in  the  denominator  of  the  dependent  variable,  as  shown  by  the  following 
equation: 


CND  burden  -  failure  rate  +  CND  rate 

For  CND  rate,  failure  rate  is  the  most  significant  Independent  variable  in 
the  prediction  equation.  Because  of  this  difference,  dissimilar  terms 
might  be  expected.  For  example,  THERMAL  and  Interactions  of  THERMAL  with 
other  variables  correlated  highly  with  CND  burden  and  CND  rate.  Yet  in 
the  presence  of  the  failure  rate  term.  THERMAL  variables  became  insigni¬ 
ficant  in  the  CND  rate  equation  and  therefore  were  not  included  in 
predicting  CND  rate, 

5.5  ANALYSIS  OF  IL,  ESTIMATOR  FOR  FFI 
5.5.1  Variables 

Variables  considered  for  use  in  predicting  IL  fell  into  the  same 
three  classifications  as  those  for  CND  burden  and  CND  rate:  initial 
design  variables,  combinations  of  initial  variables,  and  variables 
synthesized  by  regression.  Initial  design  variables  include,  in  addition 
to  the  counts  and  measures  found  on  the  worksheet  in  Appendix  B,  several 
STAMPe  measures.  These  include  number  of  tests,  number  of  failure  mode 
components,  number  of  input  signals,  and  test  leverage.  Although  "parts" 


generally  refers  to  hardware,  STAMP*  failure  mode  components  are  regarded 
as  designated  functional  units  of  a  system  indicating  a  desired  level  of 
fault  isolation.  Failure  mode  components  may  be  parts.  LRUs,  or  failure 
modes  of  parts  depending  on  the  desired  analysis.  Tests,  cotiiponents,  and 
input  signals  will  be  described  further  in  Chapter  Six.  Appendix  C  lists 
the  initial  design  variables  for  IL. 

The  majority  of  the  combined  variables  and  synthesized  variables 
consists  of  multiplicative  and  additive  combinations  of  the  STAMP* 
measures.  Combined  variables  Include  tests  divided  by  components,  tests 
divided  by  parts,  input  divided  by  components,  and  a  few  others.  Synthes¬ 
ized  variables  Include  a  topological  complexity  variable  and  its  combin¬ 
ations  with  other  variables.  (For  a  short  discussion  on  the  synthesis  of 
a  variable  using  regression,  refer  to  subsection  5.4.1.)  Appendix  C 
contains  lists  of  both  combined  variables  and  synthesized  variables  for  IL. 


5.5.2  The  Prediction  Equation 


The  equation  developed  to  predict  IL  is  as  follows: 


IL  » 


'  0 

•  Fctn 
1 


IF  Fctn<0 
IF  0<Fctn<l 
IF  KFctn 


tdiere: 

Fctn  =  0.615  -  2.48*10"8*TC  +  0.218*TEST.CMP  +  0.278* INPUT. CMP. 


The  variables  included  in  this  equation  cover  a  number  of  influences  on 
effective  fault  isolation.  The  following  descriptions  of  each  variable 
suggest  the  impact  on  isolation  level. 


TC  -  The  variable  TC  is  a  measure  of  the  topological  complexity 
of  the  LRU.  Similar  to  the  topological  complexity  measure  for 
CND  rate,  TC  approximates  the  effect  of  a  sneak  circuit  analysis 
by  compounding  the  number  of  interconnects  with  the  type  of 
components  that  act  as  switching  devices.  For  this  data  set,  the 
switching  devices  that  were  significant  Include  ICs,  relays,  and 
transistors.  Background  for  this  variable  can  be  found  in 
subsection  3.4.3. 
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TEST. CMP  -  TEST. CMP  Is  the  ratio  of  number  of  tests  to  number  of 
components.  As  this  ratio  Increases  (assuming  that  the  tests  are 
reasonably  placed  and  not.  all  testing  the  same  component),  the 
likelihood  of  a  component  having  a  test  that  identifies  it  when 
it  fails  also  increases.  Subsection  3.4.3  discusses  the  impact 
of  tests  and  caaponents  on  IL. 

INPUT. CMP  -  In  STAMP*,  many  of  the  components  and  the  input 
signals  are  functional.  Thus,  INPUT. CMP  can  be  viewed  as  the 
ratio  of  functions  to  components.  As  the  average  number  of 
components  assigned  to  a  function  increases,  I.NPUT.CMP  decreases 
and  ability  to  fault  isolate  decreases.  Subsection  3.4.3 
suggests  the  Influence  of  functional  complexity  on  IL. 


A  summary  of  the  regression  analysis  for  IL  is  presented  in  Table  5-7. 

5.5.3  Validation  of  Eouation  for  IL 

One  system  was  withheld  from  the  regression  analysis  for  IL  to  be 
used  as  a  verifier  of  the  predictive  ability  of  the  equation.  This  syst^ 
has  a  STAMPe-generated  IL  value  of  0.6107  and  a  predicted  value  of  0.6966. 
This  result  and  the  corresponding  prediction  interval  is  presented 
graphically  in  Figure  5-5,  and  the  data  are  provided  in  Table  5-8.  This  • 
verification  was  considered  good. 


5.5.4  Final  Equation  for  IL 

After  the  predictive  ability  of  the  equation  was  verified,  the  vali¬ 
dation  system  was  folded  back  into  the  data  set  to  compute  a  final  form  of 
the  equation  using  all  the  STAMP*  system  data  compiled.  Note  that  the 
form  of  the  equation  and  the  choice  of  independent  variables  remain 
fixed.  Inclusion  of  the  additional  system  is  only  to  make  minor  adjust¬ 
ments  in  the  constant  and  coefficients.  The  resulting  equation  is: 


IL 


0 

•  Fctn 
.1 


IF  Fctn<0 
IF  OSFctn^l 
IF  l<Fctn 


where : 


Fctn  =  0.590  -  2.41*10"8*TC  +  0.237*TEST.CMP  +  0.291*INPUT.CMP. 


TABLE  5-7 


EQUATION  AND  STATISTICS  FOR  IL 


a.  EQUATION  SUmART 

Variable 

Coefficient 

Std.  Error 

T-Value 

Computation 

CONSTANT 

0.6151277 

9.240679E-02 

6.656737 

TC 

-2.480748E-08 

5.99033E-09 

-4.141254 

intcon* (lCs+150*relays-17* 

xstrs) 

TEST. CMP 

0.2180233 

0.1105966 

1.971338 

tests/components 

INPUT. CMP 

0.2780414 

0.1196387 

2.324009 

inputs/components 

b.  ANALYSIS  OF  VARIANCE 

Sources  of 

Variation  SS- 

DF 

HSE 

Regression 

0.740594 

3 

0.2468646 

Error 

0.190176 

17 

1.118682E-02 

Total  SS 

0.9307699 

20 

F  -  22.06745 
R  Squared  0.7956788 


c.  F  TEST  STATISTICS 


Regression  F  «  22.07 

a  -  0.05  0.025  0.01  0.005 

■  3.20  4.01  5.18  6.16 

99.5\<Slgnificance  Level 


d.  t  TESTS  STATISTICS 


Regression 

ABS  (t  Value)  Significance  Level 


TC 

4.14 

SL>99\ 

TEST. CMP 

1.97 

90\<SL<95\ 

INPUT. CMP 

2.32 

95%<SL<98\ 

a  ■ 

0.10 

0.05 

0.02 

O.bl 

*a/2.20  “ 

1.73 

2.09 

2.53 

2.85 

0  0.2  0.4  0.6  0.8  1.0 

Predicted  IL 
FIGURE  5-5 

SCATTER  PLOT  OF  (»SERVED  VERSUS  PREDICTED  VALUES  FOR 
IL  -  INITIAL  EQUATION 
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TABLE  5-8 

VALIDATION  OF  EQUATION  FOR 

IL 

Variable 

System 

TC 

1.509  X 

106 

TEST.  CMP 

0.4706 

INPUT. CMP 

0.05B8 

PREDICTED  VALUE 

0.6966 

ACTUAL  VALUE 

0.6107 

95%  PREDICTION  INTERVAL 

(0.4528. 

0.9405) 

Table  5-9  summarizes  the  regression  analysis  for  this  final  equation  for 
IL.  Included  in  this  analysis  is  a  scatter  plot  of  predicted  IL  values 
versus  the  STAMPe-generated  values  in  Figure  5-6. 

For  ease  in  using  this  equation  to  obtain  a  predicted  value,  a  work¬ 
sheet  indicating  the  necessary  measures  and  calculations  is  found  in 
Chapter  Six. 

5.6  ANALYSIS  OF  DP,  ESTIMATOR  FOR  FFD 
5.6.1  Variables 

Because  both  DP  and  IL  are  STAMP*  measures,  all  of  the  variables  for 
IL  were  considered  in  the  attempt  to  predict  DP  (see  Appendix  C)  .  Our 
analytical  model  suggested  a  strong  interaction  between  IL  and  DP  (see 
3.4.4  for  a  discussion  of  this  interaction.)  As  the  analysis  of  DP 
continued,  other  variables  were  computed  and  added  to  the  list  of  vari¬ 
ables  from  the  analysis  of  IL.  Of  these,  only  two  were  initial  design 
variables.  They  were  the  STAMP*  measures  false  alarm  tolerance  (FAT)  and 
the  theoretical  minimum  test  .  verage  (TLMIN).  The  r^naining  additional 
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TABLE  5-9 


FINAL  EQUATION  AND  STATISTICS  FOR  IL 


a.  EQUATION  SUniARY 

Variable 

Coefficient 

Std.  Error 

T-Value 

Computation 

CONSTANT 

0.5902138 

8.5042S1E-02 

6.940221 

TC 

-2.405566E-08 

5.83062SB-09 

-4.125743 

Intcon* (lCs+150*relays 

-17*mrs) 

TEST.CHP 

0.2372213 

0.1061973 

02.23378 

tests/coiponents 

INPUT. CHP 

0.2910916 

0.116866 

02.490815 

Inputw/components 

'  b.  ANULYSIS  OP  VMtIANCE 


Sources  of  Variation 

SS 

DP 

HSS 

Regression 

0.7958355 

3 

0.2652785 

Error 

0.1963635 

18 

1.090908E-02 

Total  SS 

F  •  24.31722 

R  Squared  0.8020927 

0.9921989 

21 

C.  F  TEST  STATISTICS 


Regression  P  ■  24.32  J 

«  •  0.05  0.025  0.01  0.005 

F3.I8  •  3.16  3.95  5.09  6.03 

99.5A<Slgnlflcance  Level 


d.  t  TESTS  STATISTICS 


Regression 

ABS  (t  Value)  Significance  Level 


TC  4.13  SL>99\ 

TEST.CKP  2.23  95%<SL<98% 

INPUT. CMP  2.49  95%<SL<98\ 


a  ■ 

to/2.21  " 


0.10  0.05  0.02  0.01 
1.72  2.98  2.52  2.83 


4  Data  Points 


Predicted  IL 
FIGURE  5-6 

SCATTER  PLOT  OF  OBSERVED  VERSUS  PREDICTED  VALUES  FOR 
IL  -  FINAL  EQUATION 


variables  were  either  coint)lnatlons  and  functions  of  the  STAMP*  measures 
and  Initial  variables  or  a  synthesis  created  by  regression.  These 
variables  are  listed  In  Appendix  C. 


5.6.2  The  Prediction  Ecpiation 

The  equation  developed  to  predict  DP  is  as  follows: 

(0  IF  Fctn<0 

Fctn  IF  OSFctnSl 

1  IF  KFctn 

tdtere: 

Fctn  »  1.04  >  3.39*10‘5*oUTMEAS  -  4.64*ILFACTOR  -  0.036*FATFACTOR. 


The  variables  included  in  this  equation  cover  various  aspects  of  fault 
detection.  The  description  of  each  variable  suggests  its  inclusion  in  the 
regression  equation. 


OUTHEAS  -  A  synthesized  variable  that  approximates  the  effect  of 
the  number  of  output  signals.  Detection  is  maximized  if  the 
majority  of  the  tests  are  located  at  the  system's  functional 
bottlenecks,  which  commonly  occur  at  output.  Thus  OUTMEAS  can  be 
viewed  as  a  measure  of  functional  topology. 

ILFACTOR  -  1-IL  represents  the  percentage  of  components  that 
cannot  be  isolated,  and  ILFACTOR  influences  nondetection.  (The 
number  of  components  is  an  inverse  complication  factor  and  thus 
is  In  the  denominator.) 

FATFACTOR  -  This  variable  is  a  measure  that  describes  the  test 
topology.  FAT  can  be  expressed  as  the  average  number  of  down¬ 
stream  verifiers  for  a  given  test.  A  verifier  is  any  test  that 
is  expected  to  test  "bad"  given  a  bad  outcome  of  the  test  that  it 
is  verifying.  Verifiers  are  determined  by  extensive  functional 
analysis  of  the  system  under  test.  The  procedure  is  explained 
further  In  Chapter  Six.  FATFACTOR  is  the  sixth  root  of  the 
inverse  of  this  measure. 


All  three  of  these  measures  relate  to  the  analysis  from  subsection  3.4.4. 
A  summary  of  the  regression  analysis  for  DP  Is  presented  in  Table  5-10. 


5-28 


I f v«nAi\A-mjk  fsji m "ji  V' 


TABLE  5-10 


EQUATION  AND  STATISTICS  FOR  LP 


a.  EQUATION  SUIKART 

Variable 

Coefficient 

Std.  Error 

T-Value 

Coeiputatlon 

CONSTANT 

OUTMEAS 

ILFACTOR 

FATFACTOR 

1.042823 

-3.387432E-05 

-4.643569 

-3.628183E-02 

2.904382E-02 

1.313741E-05 

1.725775 

2.144648E-02 

35.90515 

-2.578463 

-2.690716 

-1.691738 

IO-3*lnputs 
( 1 -IL ) /coaponents 

(FAT) “1^6 

b.  ANALYSIS  OF  VARIANCE 


Sources  of  Variation 

SS  . 

OF 

MSE 

Regression 

1.1280D6E-02 

3 

3.76002E-03 

Error 

1.644135E-02 

30 

5.480449E-04 

Total  SS 

2.772141E-02 

33 

F  •  6.860789 

R  Squared  0,406908 

c. 

F  TEST  STATISTICS 

R^grcrsslon  P  *>  6* 

86 

a  ■ 

0.05  0.01 

0.025  0.005 

*^3.30  * 

2.92  3.59 

4.51  5.24 

99.5%<Signlflcance  Level 


d.  t  TESTS  STATISTICS 


Regression 

ABS  (t  Value)  Significance  Level 


OUTNEAS  2.58 
ILFACTOR  2.69 
FATFACTOR  1.69 


98%<SL<99\ 

98%<SL<99% 

90%<SL<95% 


a 

V2.30 


0.10  0.05  0.02  0.01 
1.69  2.04  2.45  2.74 


5.6.3  Validation  of  Equation  for  DP 


As  mentioned  before,  one  syst«n  was  withheld  from  the  regression 
analysis  for  DP  to  be  used  as  a  verifier  of  the  predictive  ability  of  the 
equation.  This  system  has  a  STAMPm-generated  DP  value  of  1.0  and  a 
predicted  value  of  0.8784.  This  result  and  the  corresponding  prediction 
Interval  is  presented  graphically  In  Figure  5-7.  and  the  data  are  listed 
In  Table  5-11.  The  validation  for  DP  Is  considered  poor. 


TABLE  5-11 

VALIDATION  OF  EQUATION  FOR  DP 

Variable 

System 

OUTMEAS 

93 

1-IL.CMP 

0.0229 

FAT-.,  167 

1.515 

PREDICTED  VALUE  0.8784 

ACTUAL  VALUE  1.0 

95%  PREDICTION  INTERVAL  (0.7935.  0.9632) 


5.6.4  Final  Equation  for  DP 

At  this  point,  the  validation  system  was  folded  back  into  the  data 
set  to  compute  a  final  form  of  the  equation  using  all  the  STAMP*  system 
data  compiled.  The  form  of  the  equation  and  the  choice  of  independent 
variables  remain  fixed.  Note  that  the  coefficient  for  ILFACTCW  changed 
dramatically  with  the  Inclusion  of  this  one  system.  The  resulting  equation 
Is: 


DP 


0  IF  Fctn<0 

Fctn  IF  0<Fctn<l 

1  IF  KFctn 


where: 

Fctn  =  1.03  -  3.12*10"5*OUTMEAS  -  0 .61*ILFACTOR  ~  0.035*FATFACTOR. 
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h.4 


Actual  DP 


23  Data  Points 


Predicted  DP 


FIGURE  5-7 


SCATTER  PLOT  OF  OTSERVED  VERSUS  PREDICTED  VALUES  FOR 
DP  -  INITIAL  EQUATION 
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Table  5-12  sununarizes  the  regression  analysis  for  this  final  equation  for 
OP.  Included  in  this  analysis  is  a  scatter  plot  of  predicted  DP  values 
versus  the  STAMP*  generated  values  in  Figure  5-8.  A  worksheet  indicating 
the  necessary  measures  and  calculations  is  found  in  Chapter  Sis. 

5.7  SUMMARY 

The  development  of  prediction  equations  yielded  mixed  results.  Some 
predictors  were  excellent,  and  some  were  poor.  Table  5-13  shows  the 
results  obtained  for  the  four  predicted  testability  attributes.  The 
detection  percentage  estimator  for  FFD  is  not  predictable  enough  with  the 
current  assignment  of  parameters  to  serve  any  useful  purpose  other  than 
exploratory  research . 
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TABLE  5-12 


FINAL  EQUATION  AND  STATISTICS  FOR  DP 


a.  EQUATION  StMfART 


Variable 

Coefficient 

Std.  Error 

T-Value 

Coaputatlon 

CONSTANT 

OUTMEAS 

ILFACTOR 

FATFACTOR 

1.033635 

-3.116032E-05 

-0.6056138 

-3.462438E-02 

3.190994E-02 

1 .44d685E-05 

1.05759 

2.370456E-02 

32.39224 

-2.150939 

-0.5726359 

-1.460664 

10-3*inputs 

U-IL) /conconents 
(FAT) "1/6 

b.  ANALYSIS  OF  VARIANCE 

Sources  of  Variation  SS  OF 

tSE 

Regression 

7.25174E-03 

3 

2.4n247E-03 

Error 

2.077103E-02 

31 

6.700331E-04 

Total  SS 

2.802277E-02 

34 

F  «  3.607652 
R  Squared  0.2567803 


c.  F  TEST  STATISTICS 


Regression  F  *  3.61 
«  • 

P3.3I  ■ 

97.5%<Signlficance  Level<99% 


0.05 

2.91 


0.025 

3.56 


0.01 

4.48 


0.005 

5.21 


.  t  TESTS  STATISTICS 


Regression 

ABS  (t  Value)  Significance  Level 


OCTHEAS 

2.15 

95%<SL<96A 

ILFACrOR 

0.57 

INSIGNIFICANT 

F71TFACTOR 

1.46 

80%<SL<90% 

a  “ 

0.20 

0.10 

0.05  0.02 

V2.3O  “ 

1.31 

1.69 

2.04  2.45 
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Actual  DP 


23  Data  Points 


0.80  0.84  0.88  0.92  0.96  1.00 

Predicted  DP 


FIGURE  5-8 

SCATTER  PLOT  OF  OBSERVED  VERSUS  PREDICTED  VALUES  FOR 
DP  -  FINAl.  EQUATION 


TABLE  5-13 


RESULTS  OP  THE  ANALYSIS 


Testability 

Attributes 

Estimator 

r2 

Validation 

Results 

Recommendations 

FFA 

CND  burden 

0.6039 

adequate 

Equation  useful  for 
analysis  purposes  but  not 
reconaended  for  design 
CMpl lance  because  of  low 
r2. 

FAR 

CND  rate 

0.9144 

excellent 

Shows  great  promise  for 
use  in  analysis  and 
specification  compliance. 

FFI 

IL 

0.8021 

good 

Shows  promise  for  use  in 
analysis  and  prediction. 

FFD 

DP 

0.2589 

poor 

Not  reconmended  for 
general  use. 
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OOPTER  SIX 


APPLICATIONS 


6.1  INTRODUCTION 

Several  theoretical  considerations  (Chapter  Three)  led  to  the  final 
form  of  the  prediction  equations,  and  coo^uter-based  statistical  regression 
analyses  (Chapter  Five)  yielded  the  actual  equations.  In  this  chapter, 
the  advantages  and  shortcomings  of  the  various  prediction  equations  and  a 
step-by-step  procedure  for  applying  them  are  described. 

6.1.1  Genera.’  Notes  on  Application 

The  equations  of  Chapter  Five  and  their  respective  implementation 
procedures,  described  in  the  following  sections,  vary  widely  in  their 
accuracy  as  predictors.  The  equations  may  be  ranked  In  terms  of  accuracy 
of  prediction  as  follows: 

-  CND  rate 

-  IL 

-  CND  burden 

-  DP 

Three  points  must  be  considered  tdien  evaluating  the  benefits  of  using 
prediction  equations: 

-  The  prediction  equations  are  not  exact. 

-  The  prediction  equations  are  limited  by  the  quality  and  domain  of 
the  data  used  as  Independent  variables. 
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'  The  prediction  equations  provide  predictions  for  surrogate  measure 
testability  parameters. 

Prediction  eouations  are  not  exact:  A  prediction  interval  with  an 
associated  confidence  level  (called  a  confidence  interval  estimate)  is  of 
more  use  than  the  point  estimate  provided  by  the  equation.  The  prediction 
interval  provides  a  measure  of  the  relative  precision  that  can  be  ascribed 
to  a  predicted  value.  An  approximation  of  this  interval  can  be  made  by 
using  the  standard  error  of  estimates,  s.  provided  in  Table  6-1.  This 
table  gives  the  final  equations  derived  in  Chapter  Five  and  data  necessary 
to  compute  the  estimated  prediction  interval.*  (L.  U)  from  the  point 
estimate.  P. 

For  IL  and  OP.  the  95  percent  prediction  Interval  (L.  U)  can  be 
computed  using  the  point  estimate.  P.  from  the  predicted  equation  and  the 
standard  error  of  estimate,  s. 

L  =  Max  (P  -  2s.  0) 

U  «  Min  (P  +  2s.  1). 

vihere  s  is  given  in  Table  6-1.  Each  of  the  limits  will  lie  between  0 
and  1.  For  CND  burden,  the  prediction  interval  (L,  U)  can  be  computed  by: 

L  =  Max  (P  -  2s.  0) 

U  -  Min  (P  +  2s.  100). 


*The  prediction  intervals  are  approximate,  and  they  are  based  on  the 
assumption  that  all  the  prediction  parameters  for  a  given  prediction 
application  are  at  the  mean  values  of  the  data  set  used  in  the  regression 
analysis.  Under  any  other  conditions,  the  confidence  Interval  will  be 
larger.  The  computation  of  exact  limits  requires  many  additional  statis¬ 
tics  based  on  the  covariance  matrix;  these  additional  statistics  are  not 
believed  to  be  necessary  for  most  applications. 
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TMLB  6-1 


RBGReSICM  BQtATION  SKMWnr 


VarlabU 

Equation 

Standard 
Error  of 

EitlMtaa  (a) 

Corralatlon 
Factor  (r2) 

CND  BUROm 

VOLUME 

CIMCT 

ACCESS 

TON 

THRMCON 

TMMCHPLX 

LENSTH  *  HEIGHT  •  WIDTH 

SHAU  EXT.CON  ♦2*HBDIUH  EJCT.CON  O*  LARGE  EXT.CQN 

VOLUME  *  S00*CNNCT 

FUNCTIONAL  CROSS  COUNT/NUMER  OF  SRUS 
(THERMAL  ♦l)*IHTEnC0MNKT8 

(THERMAL  M>*(ICl  ♦l-'IM)  -2.4*CAP)/NU«ER  OF  SRUS 

OO  BUROEN 

25 . 9-0 . 002*ACCESS  .  9*TOP4»0 . 003*THRMOON-0 . 077*TM«CMPLX 

7.67 

0.60 

CM)  RATE 

FLRRATE 

TRANSIENT 

TC7 

CM)  RATE 

(NA-CM))/LRU  OPERATING  HOURS;  NIL-STD-217  OR  RQUIVALINT 
(ICs  Ml’RELATS  f2*CAP  ♦2*RES-9*XSTRS) /NUMBER  OF  SRU« 

NUMBER  OF  INTERCONNBCTS*(ICB«30*]tSTRS-160*RRUn-960*SHITCHES) 

-0 . 0028*0 . 375*nj«ATE+2 . 6xlO-5*TRANS18HT*5 .9*10“*tC7 

e.3xio-« 

0.91 

IL 

TC 

TEST.  CMP 
INPUT. CMP 

NUBER  OF  INTBRC0NN8CTS*(ICs  *1S0*RBLAYS  -ITXStttS) 
TESTS/COMPONENTS 

INPUTS/COMPONEMTS 

IL 

0.59  -2.41xlO-**TC  ♦0.237*TEST.CHP  ♦0.291*IMPOT.CMP 

O.lOt 

0.80 

OUnSAS  10  •3*INPUTS 

ILFACTOR  (l-lD/CGMPOtBNTS 

FATFACTOR  (NUnSR  OF  TEST  VERIFIERS/ tTBSTS* (TSSTS-l) ] 

OP  1.03-3.12x10-5  ‘OUTHEAS  -0.61‘  ILFACTOR  -0.035‘FATFACTOR  0.026  0.26 


wUh  CND  burden  being  between  0  and  100  percent.  The  prediction  interval  (L. 
U)  for  CND  rate  can  be  computed  by: 


L  »  Max  (P  -  2s.  0) 

U  -  P  2s. 

with  CND  rate  being  occurrences  per  operating  hour. 

Prediction  eouatlwis  are  limited  bv  the  Quality  and  the  domain  of  the 
data  used  as  Independent  variables:  Table  6-2  presents  the  daaain  (range) 
of  the  measured  values  used  as  Independent  variables  in  the  prediction 
equations  Predictions  attempted  when  measured  values  are  not  within  the 
range  of  data  values  presmted  In  thr  table  represent  an  extrapolation 
that  is  subject  to  greater  errors  than  those  Indicated  by  the  statistical 
analysis  of  Chapter  Five. 

The  equations  provide  predlctltais  for  surrogate  measure  testability 
parameters:  Although  they  may  reasonably  predict  CND  burden.  CND  rate. 
IL.  and  OP.  In  some  Instances,  there  Is  no  guarantee  that  the  results  will 
be  reasonable  predictions  of  the  testability  attributes  FFA.  FAR.  PFI.  and 
FFD. 


One  or  more  of  the  following  typos  of  data  sources  may  have  to  be 
used  to  develop  the  values  for  the  predictor  variables; 

-  Design  detail  documentation,  including  specifications,  schematics, 
theory  of  operation,  and  parts  lists 

-  Environmental  analysis  and  design  expert  consultation 

-  MIL-HDBK-217  or  other  appropriate  failure  rate  prediction  data 

-  Inspection  of  hardware  prototypes  or  breadboards 
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TABLE  6-2 

SUMIARY  OP  VARIABLES  AND  VALUE  DOIAINS 

NMSured 

Applicable 

Definition  (Report 

Range  of 

variable 

Equation 

Section) 

Data  Values 

Functional  cross 
count 

CND  burden 

Number  of  parallel  paths 
(6.2.1. 1) 

8 

• 

47 

SRUs 

CND  burden, 
CND  rate.  IL 

Number  of  SRUs  In  the 

LRU  (6. 2. 1.2) 

4 

• 

42 

Length 

CND  burden 

Length  of  the  LRU 
(6. 2.3.1)  (inches) 

6 

* 

24.57 

Height 

Of)  burden 

Height  of  the  LRU 
(6.2.3. 1)  (Inches) 

4.03 

19.75 

Width 

CND  bur^n 

Width  of  the  LRU 
(6. 2. 3.1)  (inches) 

3.625 

31.2 

Thermal 

CND  burden 

Number  of  thermal  sensi¬ 
tivities  (6. 2.2. 4) 

0 

2 

Interconnects 

CND  burden. 
CND  rate.  IL 

SRU-LRU  interconnects 
(6. 2.2. 5) 

0 

• 

5.724 

Inputs/outputs 

DP 

Input/output  signal 
count  (6. 5. 1.1) 

17 

892 

Capacitors 

CM)  burden. 
CND  rate 

Capacitor  count 
(6. 2.2. 3) 

5 

2.410 

ics 

CM)  burden. 
CM)  rate.  IL 

'ntegrated  circuit  count 
(6. 2.2.1) 

0 

4.712 

InductCLts/ 

transformers 

CM)  burden 

Inductor/transformer 
count  (6. 2. 2. 2) 

0 

247 

Relays 

CND  rate.  IL 

Relay  count  (6.3.2) 

0 

- 

87 

Resistors 

CND  rate 

Resistor  count  (6.3.2) 

10 

- 

1543 

Switches 

CM)  rate 

Internal  switch  count 
(6.3.3) 

0 

• 

10 

Transistors 

CM)  rate.  IL 

Discrete  transistor 
cmint  (6.3.2) 

0 

291 

Small  external 
connectors 

CND  burden 

0  to  4  pin  connectors 
(6.2. 3.2) 

0 

9 

Medium  external 
ccxmectors 

CND  burden 

5  to  14  pin  connectors 
(6. 2.3. 2) 

0 

2 

Large  external 
C(»inectors 

CM)  burden 

15  and  over  pin  connec¬ 
tors  (6. 2. 3. 2) 

0 

11 

Failure  rate 

CM)  rate 

LRU  failure  rate  (6.3.1) 

0.43 

- 

34.3* 

Compcments 

IL,  DP 

Component  failure  mode 
count  (6.4.2) 

10 

■ 

235 

Input  signals 

IL,  DP 

Input  count  (6.4.3) 

4 

- 

184 

Tests 

IL,  DP 

Test  count  (6.4.2) 

10 

— 

936 

Test  verifiers 

DP 

DoMnstream  test 
verifiers  (6.5.3) 

30 

84  891 

IL 

DP 

Isolation  level  (6.5.2) 

0.1408 

1 

*Per  1,000  operating  hours. 
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6.1.2  Notations  Us»d  In  This  Chapf  r 


In  addition  to  the  various  conventions  and  notations  already  used  In 
this  document  (see  Glossary),  a  number  of  graphical  notations  are  employed 
to  Illustrate  computatlmal  algorithms  for  the  prediction  equations. 
These  notations  are  defined  In  Figure  6-1.  The  graphical  descrlptl^is  of 
the  four  prediction  procedures  are  designed  to  serve  as  templates.  Once 
appropriate  data  are  obtained,  one  should  be  able  to  compute  the  predicted 
values  by  copying  the  appropriate  figures  and  tables  and  stepping  through 
the  prediction  procedures  by  filling  In  the  blanks  on  the  figures. 

6.1.3  Information  Sources. 

Throughout  this  chapter,  suggested  documents  for  fielded  systems  are 
listed  as  sources  for  the  required  information.  The  documents  mentioned 
are  for  fielded  systems  for  two  reasons.  First,  these  were  the  sources 
used  for  this  study,  and  sec(»d.  documentation  for  fielded  systems  Is 
generally  more  standard  In  content  and  format.  For  systems  that  are  still 
In  the  design  phase,  equivalent  documentation  may  Include  such  sources  as 
circuit  diagrams,  functional  block  diagrams,  wiring  lists,  reliability 
prediction  data,  failure  modes  and  effects  analysis  (FNEA).  reliability, 
availability,  and  maintainability  documentation,  and  the  Integrated  logis¬ 
tics  support  plan  (ILSP).  Another  source  of  information  would  be  Inter¬ 
views  with  technicians  or  other  persuinel  who  have  experience  with  systems 
fielded  in  an  environment  similar  to  that  of  the  system  under  design. 

6.2  CND  BURDEN  —  ESTIMATOR  FOR  FFA 

The  prediction  equation  for  CND  burden  is  considered  adequate.  When 

possible,  the  alternative  predictor.  CND  rate  should  be  used  because  of 
2 

Its  higher  R  value.  Figure  6-2  shows  the  equation  for  predicting  CND 
Burden  and  the  four  synthesized  variables.  These  variables  are  described, 
along  with  their  synthesis  procedures,  in  the  following  subsections.  Note 
that  CND  burden  is  a  percent  measure  between  0  and  100. 
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SYMBOL 


DEFINITION 


AN  ACTIVITY 

A  VALUE.  VARIABLE  OR  CONSTANT 

b“a.>ta^xa,x‘  •  ‘xa 
12  3  n 

-b^a^a^a  ♦•••♦a 

12  3  n 

c  ■  a/b 

•  b  ■  -a 


b  -  MAX  (a^ ,  a^.  a^,  .  .  .  ,a^) 

b-MlN(a  .  a  .  a  .  ,a  ) 

1  2  3  •  •  •  n 


FIGURE  6-1 

OaiPHICAL  NOMENCLATURE  APPLIED  TO  PREDICTION  PRXEDURES 


6.2.1  Topoloqlcftl  cowpltxltv  VarlabU  (TOM) 


This  synth«slz«d  varlabl*  attoapts  to  nuatrlcally  charactorlza  tha 
coaplaxlty  Inhortnt  In  tha  topology  of  an  URU  datlgn.  (Saa  tha  discussion 
on  topological  complaxlty  In  Sactlon  3. 4. 2. 2.)  Plgura  6-3  susiarlzas  tha 
synthasls  of  TOP4.  Two  neasuranants  ara  raqulrad  for  this  varlabla  —  a 
functional  cross-sactlon  count  and  an  SRU  count.  Thasa  counts  ara 
dascribad  In  subsactions  6.2. 1.1  and  6. 2. 1.2.  and  ara  furthar  subdlvldad 
In  Flgura  6-4. 


6. 2. 1.1  Functlwial  Cross-Sactlon  Count  (CROSS  CQUMT) 

Tha  functional  cross-sactlon  coui\t  of  an  LPU  Is  naasurad  using  a 
top-laval  functional  block  diagraa  of  tha  LRU  (prasuMsd  to  ba  orlantad 
from  laft  to  right).  This  typa  of  diagram  Is  typically  found  in  Air  Porca 
0-Level  Technical  Orders.  The  naasuraKant  involvas  the  determination  of 
the  maximum  number  of  parallel  functional  paths  In  tha  diagram. 

One  procedure  for  performing  this  maasurasMMit  Is  to  scan  tha  diagram 
with  a  vertically  oriented  straight  edge  and  to  count  tha  number  of 
horizontal  paths  cut  by  the  edge.  The  edge  Is  moved  from  laft  to  right 
until  the  count  changes.  This  next  count  Is  recorded  and  the  procedure 
continues.  At  the  conclusion,  tha  maximum  number  of  paths  cut  by  the  edge 
Is  taken  as  the  functional  cross-section  count.  The  diagram  used  for  this 
measure  is  the  top  level  functional  block  diagram  illustrating  the 
relationships  between  SRUs  or  other  functional  entitles.  Bach  physically 
drawn  line  should  be  counted  once,  even  if  It  represents  a  bus  or  multiple 
signal  line. 

Caution  should  be  used  in  the  interpretation  and  use  of  this  measure. 
A  functional  block  diagram  might  be  easily  modified  to  change  the  outcome 
of  this  type  of  measurement.  Althou^  the  functional  cross-section  count 
indicates  some  measure  of  parallelism  useful  for  predicting  CND  burden,  it 
should  not  be  used  beyond  this  purpose  without  further  analysis. 
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ACCESS 


PREDICTION  PROCEDURE  FOR  CND  BURDEN 


TOP  4 


COUNT  NUMBER  OF  SRU»  ^ 
IH  LRU  (SFCTION  2.1.21 


SYNTHESIS  OF  THRMCMPLK  ftND 


6. 2. 1.2  SRU  Count  (SRUs) 


This  msasurement  involves  the  count  of  the  SRUs  that  compose  a  given 
LRU.  Not  Included  in  this  count  are  SRUs  with  only  mechanical  parts  (such 
as  gyros)  or  without  any  components  (such  as  a  mother  board).  This  type 
of  information  may  be  found  in  Air  Force  0-Level  Technical  Orders  or 
equivalent  documentation. 

6.2.2  Measures  of  Thermally  Ccnnpounded  Failures  (THRMCMPLX  and  THRMCCT!) 

The  second  and  third  variables  affecting  CND  burden,  THRNCMPLX  and 
THRMCON,  attempt  to  characterize  the  compounded  effect  of  environmentally 
Induced  thermal  stress  on  the  Intermittent  behavior  of  various  types  of 
components.  The  synthesis  procedures,  are  given  in  Figure  6-4.  The 
measurement  data  required  for  these  procedures  are  discussed  in  the 
following  subsections. 

6.2.2. 1  Integrated  Circuit  Count  (ICs) 

This  measurement  includes  both  linear  and  digital  integrated  circuits 
and  hybrid  devices.  A  count  of  the  total  number  of  these  components  used 
in  an  LRU  is  made.  The  sources  for  this  information  include  Air  Force 
Technical  Order  Illustrated  parts  breakdown  (IPBs),  schematic  diagrams,  or 
other  design  documentation. 

6. 2. 2. 2  Inductor  Count  (IND) 

The  IND  measurement  represents  the  total  number  of  inductors  in  an 
LRU  design.  This  number  includes  inductors  used  in  power  supply  filters, 
high  frequency  chokes,  and  transformers.  It  does  not  include  the  induc¬ 
tances  in  relay  coils.  Sources  for  this  information  include  IPB  documen¬ 
tation  and  schematic  diagrams. 

6. 2. 2. 3  Capacitor  Count  (CAP) 

This  measurement  is  the  total  count  of  capacitors  used  in  the  LRU. 
Information  sources  are  the  same  as  for  ICs  and  inductors. 
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6. 2. 2. 4  Thermal  Environment  (THHaiRL) 


This  variable,  THERMAL,  is  an  integer  in  the  range  0  to  3  and  is 
derived  by  completing  the  information  requested  in  the  form  shown  in 
Figure  6-5.  THERMAL  attempts  to  characterize  the  level  of  thermal  stress 
to  which  a  given  LRU  is  subject  due  to  its  environment.  The  source  of 
this  information  must  be  an  Air  Force  flight  line  maintenance  specialist, 
environmental  design  expert,  environment  simulation  program,  or  some  other 
equivalent  source. 

6. 2. 2. 5  SRU-to-LRU  Interconnects  aNTERCOMNECTS) 

This  variable,  INTERCONNECTS,  is  the  total  number  of  electrical 
interconnects  used  to  mate  all  of  the^  SRUs  to  a  given  host  LRU.  This 
number  includes  all  signals,  power,  and  grounds.  C^ly  those  interconnects 
that  carry  electrical  functions  are  used.  No  spares  are  included  in  the 
count.  These  data  are  obtained  by  analyzing  Air  Force  I-Level  Technical 
Orders,  IPBs,  or  equivalent  documentation. 

6.2.3  Accessibility  Variable  (ACCESS) 

The  ACCESS  variable  is  designed  to  measure  the  difficulty  with  which 
the  LRU  is  removed  from  the  aircraft  for  fault  investigation.  Figure  6-6 
depicts  the  computation  of  ACCESS.  VOLUME  and  CNNCT,  the  two  variables 
used  to  calculate  ACCESS,  are  discussed  in  subsections  6. 2. 3.1  and  6. 2. 3. 2. 

6. 2. 3.1  Volume  (VOL) 

The  variable  VOL  is  a  product  of  LRU  height,  width,  and  length.  The 
measures  must  be  in  inches  for  the  equation.  These  measures  are  commonly 
found  in  Air  Force  0-Level  Technical  Orders  or  equivalent  documentation. 

6. 2. 3. 2  External  Connectors  (CNNCT) 

As  seen  in  Figure  6-4,  CNNCT  is  a  linear  combination  of  small,  medium, 
and  large  external  connectors.  These  do  not  include  test  or  programming 
connectors.  Small,  medium,  and  large  are  classifications  based  on  the  pin 
count:  0  to  4  pins  constitute  a  small  connector,  5  to  14  pins  constitute 
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ica*u.)rv  m 


THKRMXL  BMVZROMMENT  CBARACTSRXZATXON 


SYSTEM: 

YES 

NO 

IS  THE  SYSTEM  LIKELY  TO  BE  SUBJECT  TO  TEM¬ 
PERATURE  EXTREMES,  DUE  TO  CLIMATE,  ALTITUDE, 
ADJACENT  SYSTEMS,  OR  COMBINATIONS  THEREOF? 

IS  THE  SYSTEM  LIKELY  TO  BE  SUBJECT  TO  A  HIGH 
DEGREE*  OF  THERMAL  CYCLING,  DUE  TO  CHANGES 

IN  CLIMATE  AND/OR  ALTITUDE,  OPERATIONAL 

PROFILES  OF  ADJACENT  SYSTEMS,-  OR  COMBI¬ 
NATIONS  THEREOF? 

IS  THE  SYSTEM  LIKELY  TO  BE  SUBJECT  TO  THERMAL 
SHOCK*  ,  DUE  TO  MISSION  CHARACTERISTICS, 

ADJACENT  SYSTEM  BEHAVIOR,  OR  COMBINATIONS 
THEREOF? 

ADD  THE  NUMBER  OF  "YES"  RESPONSES  HERE: 

THERMAL  ENVIRONMENT 

□ 

^  A  qualified  maintenance  technician  at  the  flight  line  level 
is  the  best  source  for  this  information. 


FIGURE  6-5 

THERMAL  ENVIRONMENT  CHARACTERIZATION 
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medium  connector,  and  15  and  more  pins  constitute  a  large  connector. 
CNNCT  Is  then  calculated  by  adding  the  number  of  small  external  connecters, 
twice  the  number  of  medium  connectors,  and  three  times  the  number  of  large 
connectors.  The  number  of  pins  and  connectors  may  be  found  In  Air  Force 
O-Level  Technical  Orders,  by  Inspection  of  the  LRU.  or  by  equivalent 
documentation. 

6.3  CND  RATE  —  ESTIMATE  FOR  FAR 

The  prediction  equation  for  CND  rate  Is  reasonably  accurate.  Uhen 
possible,  this  attribute  should  be  used  Instead  of  CND  burden.  However, 
as  is  the  case  with  all  models,  the  output  of  this  prediction  equation  Is 
only  as  good  as  the  measurement  data  used.  One  of  the  measures  used  to 
generate  CND  rate  Is  LRU  failure  rate.  During  system  design,  field  fail¬ 
ure  rates  may  be  unknown  and.  thus,  must  be  estimated.  If  the  estimate  Is 
very  coarse,  the  resulting  prediction  of  CND  rate  also  will  be  coarse. 
When  this  estimate  Is  known  to  be  highly  Inaccurate,  it  may  be  advisable 
to  use  CND  burden.  The  CND  rate  is  measured  In  occurrences  per  hour. 

The  CND  rate  predictor  requires  the  LRU  failure  rate  variable  and  two 
synthesized  variables.  These  are  discussed  In  the  following  subsections. 
The  procedure  for  predicting  CND  rate  is  detailed  In  Figure  6-7  and 
subdivided  further  In  Figures  6-8  and  6-9. 

6.3.1  LRU  Failure  Rates  (FLRRATE) 

If  CND  rate  is  being  predicted  for  a  syston  In  the  field,  actual 
failure  rate  data  should  be  compiled  and  used  as  the  value  for  the  variable 
FLRRATE.  Failure  rate  must  be  the  number  of  LRU  failures  per  operating 
hour.  The  failure  rates  used  In  this  study  were  calculated  as  follows: 


Failure  rate  » 


No.  of  maintenance  actions  -  no.  of  CND  events 

No.  of  aircraft  In  survey  *  no.  of  operating  hours  per 

aircraft 


This  yields  units  of  failures  per  operating  hour. 


-0.0028 


PREDICTION  PROCEDURE  FOR  CND  RATE 


COUNT  nvHBtn  or  situs 
IN  (SeCTlON  <.3 


VKRIABLE  snUTHESIS  OF  TRANSIENT 


I 


When  used  for  a  systen  In  the  design  stage,  the  failure,  rates  may 
have  to  be  estimated.  There  are  a  number  of  viable  techniques  for  this 
estimation: 

-  use  :.f  MIL-HDBK-217 

-  Vendor  and  manufacturer  estimates  of  failure  rates  based  on  testing 
and  screening  and  simulations 

-  Other  reliability  estimation  programs 

6.3.2  Tendency  for  Transient  Behavior  (TRANSIENT ) 

The  second  variable  used  to  estimate  CND  rate.  TRANSIENT,  attempts  to 
characterize  the  tendency  of  an  LRU  to  exhibit  intermittent  fal lures 
resulting  from  marginal  or  degrading  components.  This  synthesis  procedure 
is  depicted  In  Figure  6-8.  The  measurements  required  are  available  from 
All  Force  Technical  Orders  (I-Level  and  IPBs)  or  a'qulvalent  documentation. 
The  measurements  are  the  following: 

-  RELAYS:  The  total  number  of  relays  in  an  LRU 

-  CAPACITORS:  The  total  number  of  capacitors  used  in  an  LRU 

-  RESISTORS:  The  total  nwber  of  resistors,  both  fixed  and  variable, 
in  an  LRU 

-  TRANSISTORS:  The  total  number  of  discrete  transistors,  including 
FBTs.  BIPOLAR.  SCRs.  and  TRIACs.  etc.,  that  are  In  an  LRU  design 

-  INTEGRATED  CIRCUITS:  The  total  number  of  ICs  in  an  LRU  (see 
Section  6. 2. 2.1) 

-  SRUs:  The  total  number  of  SRUs  that  compose  a  LRU  (see  Section 

6. 2. 1.2) 


6.3.3  Topological  Complexity  (TC7) 

TC7  numerically  characterizes  the  likelihood  of  a  sneak  circuit 
existing  in  an  LRU.  For  more  details,  see  the  discussion  of  topological 
complexity  In  Section  3.4.)  The  equation  for  synthesizing  this  variable 
Is  shown  In  Figure  6-9.  TC7  requires  three  of  the  same  variables  that 
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TRANSIENT  requires  —  RELAYS.  ICs.  and  TRANSISTORS.  In  addition,  thara 
ara  t%io  othar  varlablas.  SWITQCS  and  INTERCONNECTS. 

SWITCHES  Is  a  count  of  tha  nuNMr  of  swltchas  In  tha  LRU  daslgn. 
This  count  may  ba  obtained  by  ravlawlng  Air  Porca  Tachnlcal  Ordars  (I-Laval 
and  IRBs)  or  equivalent  documentation. 

INTERCONNECTS  Is  the  total  number  of  electrical  Intarccffuiacts  used  to 
mate  SRUs  to  a  host  LRU.  This  maasurammt  Is  described  In  Section  6. 2. 2. 5. 

6.4  IL  —  ESTIMATOR  FOR  FFJ 

The  predictor  equation  for  IL  Is  reasonably  accurate;  however.  IL  Is 
not  FPI.  only  an  upper  bound.  Thus,  the  resulting  value  obtained  from 
this  procedure  will  represent  an  estimate  of  the  upper  limit  of  FFI  as 
discussed  in  Sectlor.  2.2.  The  procedure  for  estimating  IL  Is  described  In 
Figures  6-10  through  6-12.  The  IL  predictor  uses  three  synthesized  vari¬ 
ables  as  discussed  in  the  following  subsections.  The  IL  Is  a  normalized 
value  bet%ieen  zero  and  one  with  no  units. 

6.4.1  Topological  Complexity  (TC) 

Figure  6-11  depicts  the  synthesis  of  TC.  This  variable  Is  similar  to 
the  variable  TC7  used  In  the  predictor  equation  for  CND  rate.  A  discussion 
of  the  measured  data  Is  given  in  Section  6.3.2. 

6.4.2  Test  System  and  Functional  Characteristic  (TEST. CMP) 

Figure  6-12  depicts  the  synthesis  of  TEST. CMP.  This  consists  of  two 
design  characteristics  (tests  and  failure  mode  components). 


Tests*  will  include  the  sum  total  of  signals,  indijatlons.  or  other  | 
observable  events  that  may  be  either  an  output  of  the  syst«a  or  caused  to  i 
happen  during  a  test  procedure.  Failure  mode  components  are  regarded  as  S 


*For  definition  of  this  term,  see  Appendix  D. 
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PREDICTION  PROCEDURE  FOR  IL 


COUNT  NUMBER  OE  ICs 
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VARIABLE  SYNTHESIS  FOR  TC 


COUNT  THE  NUMBER  OF  TEST 
IN  LRU  (SECTION  6.4.1) 


VARIABLE  SYNTHESIS  FOR  TEST. CMP  AND  INPUT. CMP 


designated  functional  units  of  a  system  indicating  a  desired  level  of 
fault  isolation.  They  may  be  parts,  LRUs,  or  failure  modes  of  parts 
depending  on  the  desired  analysis.  (The  total  number  of  failure  conclu¬ 
sions  that  could  be  reached  based  on  different  failures  that  could  happen 
to  the  system  is  components.) 

6.4.3  Test  System  and  Functional  Characteristic  (INPUT. CMP) 

Figure  6-12  depicts  the  synthesis  of  INPUT. CHP.  This  consists  of  two 
design  characteristics  (functional  input  signals  and  failure  mode 
components).  Functional  input  signals  consist  of  all  input  representing  a 
function,  where  an  input  is  defined  as  any  active  electrical  signal 
including  power  and  ground  but  not  Including  spares.  For  example,  a  bus 
line  with  several  signals  but  only  one  function  would  be  considered  as  one 
functional  input  signal. 

6.5  DP  —  ESTIMATOR  FOR  FFD 

The  prediction  equation  for  DP  Is  poor  In  terms  of  accuracy.  Its  use 
Is  not  recommended.  The  validity  of  any  estimates  derived  using  this 
equation  Is  further  compromised  due  to  use  of  the  Independent  variable,  IL. 
(IL  is  an  estimate  itself.)  The  dependence  of  detection  on  isolation  (and 
vice  versa)  is  a  confirmation  of  the  emalytical  work  for  these  terms  as 
discussed  in  Chapter  Three. 

The  procedure  for  prediction  of  DP  is  described  in  Figures  6-13  and 
6-14.  The  variables  used  In  the  prediction  of  DP  are  discussed  in  the 
following  sections.  DP  is  a  normalized  value  ranging  between  zero  and  one 
with  no  units. 

6.5.1  Output  Measure  (OUTHEAS) 

The  first  variable  in  the  equation  for  DP,  OUTMEA3,  approximates  the 
number  of  functional  output  signals  in  a  system.  Its  synthesis  is  detailed 
in  Figure  6-13.  Two  variables,  10  and  INPUTS,  are  used  to  calculate 
OUTMEAS. 


PREDICTION  PROCEDURE  FOR  DP 


VARIABLE  SYNTHESIS  OF  OUTHEAS  AND  ILFACT(» 


6. 5. 1.1  Input/Output  (IQ) 

This  variable  measures  the  actual  Input  and  output  signals  to  and 
from  the  LRU.  10  Is  the  number  of  active  pins  In  the  external  connectors. 
This  active  pin  count  can  be  obtained  from  Air  Force  I-Level  Technical 
Orders  or  equivalent  documentation. 

6. 5. 1.2  Input  Signals  (INPUT. CMP) 

This  variable  is  described  In  Section  6.4.3. 

6.5.2  IL  Component  (ILFACT(«) 

The  elements  necessary  to  compute  this  factor  are  described  In 
Section  6.4. 

6.5.3  Test  Redundancy  Measure  (FATFACTOR) 

The  synthesized  variable  FATFACTOR  Is  derived  from  a  count  of  the 
number  of  tests  (described  In  Section  6.4.2}  that  maice  up  the  test  system 
and  a  count  of  the  total  number  of  test  verifiers  within  that  system  as  in 
Figure  6-15.  For  a  given  test,  t^,  and  another  test,  t^  if  we  expect 
tj  to  test  "bad”  when  t^  tests  "bad,"  t^  is  said  to  verify  t^.  All 
t^'s  that  exhibit  this  behavior  are  said  to  be  test  verifiers  of  t^. 
Verifiers  are  determined  by  extensive  functional  analysis.  A  table  for 
collecting  data  for  the  computation  is  shown  in  Figure  6-16. 
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VARIABLE  SYNTHESIS  FOR  PATFACTOR 
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CfAPTBR  SEVEN 


SUHHARy.  CONCLUSIONS.  AND  RECOHNENDATIONS 


7.1  REVIEW  OF  RESULTS 

This  reseai'ch  yielded  a  nutnber  of  useful  Insights  Into  field-level 
testability.  The  data  gathered  and  analyzed  during  this  project  represent 
a  reasonable  start  on  the  problem  of  predlctlm  of  field-level  testability 
attributes  for  complex  electronic  systems.  The  correlation  for  CND  rate 
(an  estimator  for  FAR)  and  Isolation  level  (an  estimator  for  FFl)  are  both 
sufficient  to  begin  some  predictive  «K>rk  during  design  phases.  The  vali¬ 
dation  of  both  of  these  predictors  Is  believed  to  be  sufficient  for  use  as 
estimators  for  design  compliance.  There  Is  some  concern  that  a  CND  rate 
prediction  is  tied  to  the  accuracy  of  a  failure  rate  prediction.  Failure 
rate  has  been  difficult  to  predict  In  the  past.  Several  options  are 
available  for  predicting  this  value.  The  best  generally  available  source 
of  Information  Is  MIL-HDBK-217.  The  correlation  of  ca®  rate  Is  good 
enough  that  we  can  state  that  the  accuracy  of  prediction  of  CND  rate  will 
be  roughly  equal  to  the  accuracy  of  prediction  of  failure  rate.  Prediction 
of  CND  burden  Is  marginal  at  this  time  for  design  compliance,  but  may  be 
useful  In  a  design  review  process  as  a  flag  pointing  to  potential  problem 
areas.  If  an  accurate  failure  rate  prediction  Is  available,  it  may  be 
better  to  estimate  CND  burden  using  CND  rate  prediction  rather  than  using 
the  CND  burden  prediction.  This  can  be  accomplished  by 


Detection  percentage  has  not  been  well  enough  defined  (In  a  mathematical 
sense)  for  any  use  at  this  time,  except  for  exploratory  purposes. 
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7.2  IMPROVING  THE  PREDICTIONS 


7.2.1  increasing  Data  Samples 

The  data  fomts  In  Appendix  h  provide  the  basis  for  achieving  a 
statistically  more  significant  data  base  for  future  «foric.  The  completion 
of  data  forms,  such  as  those  found  In  Appendix  B  could  be  made  a  contract 
requirement  for  any  new  system  purchased.  Further,  one  may  want  to  modify 
such  a  form  to  reflect  a  higher  content  of  the  “wish  list"  given  in  Appen¬ 
dix  A.  After  a  sufficiently  large  sample  of  these  modified  design 
description  data  forms  and  similar  data  from  the  AFM  56-1  data  gathering 
systmn  are  gathered,  the  analyses  presented  In  Chapter  Five  can  be  redone 
to  reflect  the  Increased  robustness  of  the  design  data.  In  addition  to  a 
more  idealistic  design  variable  data  tese,  a  greater  number  of  aircraft 
types  than  that  (three)  used  for  this  study  may  prove  beneficial, 
especially  if  state-of-the-art  electronic  systems  are  represented. 

7.2.2  More  Precise  Field  Data 

The  field  data  gathered  during  these  efforts  were  well  within  the 
statistically  significant  range  and  only  limited  additional  gains  can  be 
made  with  Increased  sample  size.  Using  Improved  field  data,  as  outlined 
in  the  Phase  I  Report,^  especially  In  the  area  of  how  faults  were 
detected  and  how  they  were  fault  isolated,  would  Improve  predictions.  By 
being  able  to  categorize  "how"  questions,  the  set  theory  model  shows  that 
a  more  precise  delineation  of  field  data  Is  oossible.  This  would  reduce 
the  need  for  the  large  number  of  surrogate  measures  used  for  this  work. 
Improved  measurability  may  also  be  the  key  to  the  problem  of  predicting 
detectability  (FFD),  tdilch  has  thus  far  eluded  us. 

7.3  ADDITIONAL  AREAS  OF  RESEARCH 

It  may  be  desirable  to  conduct  further  research  on  the  Impact  of  this 
work  In  testability  predictions  on  other  areas.  Two  such  areas  Identified 
are: 


-  Life-Cycle-Cost  Analyses 

-  Maintenance  and  Readiness  Analyses 
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The  iM.lor  influence  of  failure  rate  cm  CND  rate  indicates  yet  another 
reason  to  specify  high  reliability.  Field  false-alara  rate  predictions 
(estimated  by  OS)  rate)  should  be  Included  In  life-cycle  cost  analysis  of 
reliability  goals  and  standards. 

The  Inherent  relationship  between  maintenance  and  readiness  was 
demonstrated  during  the  Phase  1  field  Maintenance  model  development.  The 
flow  model  offers  a  method  by  which  these  relationships  may  actually  be 
quantified.  The  data  necessary  to  apply  the  flow  model  to  an  actual 
maintenance  system  Is  both  large  and  labor-intensive,  but  may  well  be 
worth  the  effort. 

7.4  RECOMfENDATIONS 

The  following  actions  are  recommended: 

-  Develop  a  specification  procedure  for  field  CND  rate  as  opposed  to 
field  false-alarm  rate.  This  would  provide  a  specified  field 
testability  parameter  that  is  both  field  iwasurable  and 
predictable. 

-  Develop  a  specification  procedure  based  upon  the  Isolation-level 
parameter  to  provide  a  specified  field  parameter  that  Is 
predictable. 

-  Use  CND  rate  and  IL  predictions  during  preliminary  and  critical 
design  reviews  as  the  basis  for  verifying  or  requiring  manufac¬ 
turer  improvements  in  system  testability. 

-  Use  CND  burden  predictions  as  early  indicators  of  potential 
maintenance  probl^ns. 

If  a  working  prediction  procedure  for  FFD  is  to  be  achieved,  modifi¬ 
cations  to  the  AFTO-349  maintenance  data  collection  system  will  need  to  be 
made  or  a  special  study  undertaken  to  measure  the  necessary  information  as 
suggested  in  Phase  I.  This  includes  reporting  the  origin  of  the  mainte¬ 
nance  activity  (pilot.  BIT,  etc.)  and  the  basis  of  the  resolution  of  the 
activity  (if  NSM  was  sufficient).  It  is  believed  that  such  information 
would  also  Improve  the  prediction  procedure  for  FFI  and  remove  the  depen¬ 
dency  of  both  FFD  and  FFI  on  STAMPe-related  measures. 
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UNRESTRICTED  VARIABLES  FOR  DEVELOPMENT 
OF  PREDICTION  EQUATIONS 


COMPONENT  CHARACTERISTICS 
Actuators 
Ninbar  Used 

Actuatlm  Medium  (Rotational 
Mechanical.  Acoustic,  etc.) 
Ratings 
Output  Ranges 
Input  Requirements 
Power  Dissipation 
Electrical  Impedance 
Nominal 
Minimum 
Maximum 
Utilization 
Loading 

Frequency  of  Use 
Duty  Cycle 

Principle  of  Operation 

Capacltors-Flxed  Value 

Number  Used 
Rating 
Capacitance 
Voltage 
Discharge  Rate 
Usage 
Voltage 
Mean 
Variance 
Spectra 
Current 
Mean 
Variance 
Construction 
Materials 
Packaging 


Capacltors-varldble 

Number  Used 
Rating 
Capacitance 
Voltage 
Discharge  Rate 
Usage 
Voltage 
Mean 
Variance 
Spectra 
Current 
Mean 
Variance 
Construction 
Materials 
Packaging 
Mechanical  Action 
Mechanical  Usage  Profile 

Circuit  Boards;  Wire  Wrap 
and  Multiwire 

Number  Used 
Construction 
Materials 
Bonding  Techniques 
Traces 

Topological  Complexity 
Densities 
Solder  Technique 
Level  of  Quality  Ccaitrol 
Component  Mounting 
Number  Layers 

Number  Through-hole  Connections 
Board  Mounting  to  Connectors/ 
Frame 
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apd  WumwUt  (contlnuad) 

ThtnMl  Conductivity 
Coatings 

NiMber  Used 
consttMction 
Packaging 
Contact  Materials 
Action 
Toggle 
Push  Button 
Rating 
voltage 
CurrMit 
Utilization 
Voltage 
Nominal 
Peak 
Current 
Nominal 
Peak 

ceamectors  (Including  On-Board 
connectors) 

Number  Used 
Number  of  Pins 
construction 
Type 
Edge 

Pin  and  Socket 
Materials 
Contact 
package 
Mating  Depth 
Actlcm 

Zero  Insertion  Force 

isolation  of  Contact  Area 
from  Environment 
Insertion/Extraction  Force 
Alignment  Sensitivity 
Matlng/Dematlng  Technique 
Manual 

Semiautomatic 
Automatic 
Tool  Assisted 


Connectors  (Including  Qn-Board 
Connectors)  (continued) 

Pressure 

Pressure  Lock 
Screw  Lock 
Ratings 
Voltage 
Current 
Frequency 
Utilization 
Electrical 
Mechanical 

Number  Used 

Frequency 

Precision 

Environmental  Support 
Packaging 
Power 
Rated 

Utlllzatl(» 

Diodes 
Number  Used 

Type  (Zener,  Pin,  Varactor,  etc.) 
Application  (Switching, 

Power,  etc.) 

Rating 
Forward  Bias 
Peak  Current 
Maximum  Current  -*  Time 
Voltage  Drop 
Capacitance 

Reverse  Bias  Peak  Voltage 
Utilization 
Forward  Bias 
Peak  Current 
Nominal  Current 
Reverse  Bias  Peak  Voltage 
Frequency  Spectra 

Indicators 

Number  Used 
Type 

Incandescent 
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Indicators  (cmtlnuad) 

Noon 

LED 

LCD 

Ratings 

Voltaga 

Currant 

Frequency 

Utilization  Duty  Cycle 
Construction 
Packaging 
Mounting 

Inductors 

NuaE>er  Used 
Ratings 
Inductance 

Volt-aapere  Capability 
Const ructl<m 
Core 

Material 

Structure 

Shape 

Dimensions 

Variability 

Windings 

Wire  Gauge  and  Type 
Number  of  Turns 
Average  Winding  Radius 
Mounting 
Utilization 
Spectra 
Voltage 
Peak 
Nominal 
Mechanical 

Integrated  Circuits 

Number  Used 

Scale  of  Integration  (SSI.  MSI. 

LSI.  VLSI.  VHSIC) 

Application  (Digital,  Linear. 
Voltage  Regulation) 

Logic  Family /Sol Id  State 
Technology  (e.g.,  TTL.  ECL. 
CMOS,  etc.) 

Bandwidth 

Capability 

Utilization 


Integrated  Circuits  (continued) 

Supply  voltage  Requirements 
Power  Consuifktlon 
Packaging 
Can 

Leadless  Chip  Carrier 
Dip 

Pln*-out 

Input 

Output 

Number 

Pan-<Ait  Capability 
Usage  Duty  Cycle 
Mounting  Technique 

Elim 

Nurt>er  Used 
Rating 
Activation 

Voltage/Current 
Time  Delay 
Coll  Inductance 
Signal 

Maximum  Voltage 
Maximum  Current 
Mlnlaui  voltage 
Minimum  Current 
Construction 
.  Packaging 
Mounting 
Utilization 
Duty  Cycle 
Switching  Rate 
Signal 

Nceinal  Voltage 
Peak  Voltage 
Nominal  (hirrent 
Peak  Current 
Operational  Design 
Solid  State 
Reed 

Normally  Open/Closed 
Moverent/Holding  Mechanism 

Reslstors-Flxed 

Number  Used 
Rating 

Resistance 

Value 


R^slstors-Flxed  (continued) 


Sensors  (continued) 


Precision 

Power 

Snail  Signal  Capacitance 
Conposltlon 
Usage 
Voltage 
Nominal 
Peak 

Frequency  Range 
Mounting  Method 
Thermal  Olssipaticm  Method 

Reslstors-Variable 

Number  Used 
Rating 

Resistance  Value 
Power 

Small  Signal  Capacitance 
Construction 
Materials 
Mechanical 
Structure 
Rotary 
Slide 
Resolution 

Usage 

Voltage 

Nominal 

Peak 

Frequency  Range 
Mechanical 
Adjustment  Rate 
Duty  Cycle 

Sensors 

Number  Used 

Sensing  Medium  (Acoustic,  Gas 
Pressure,  etc.) 

Ratings 

Input  Sensitivities 
Output  Ranges 
Power  Requirements 
Electrical  Impedance 
Nominal 
Maximum 
Minimum 


Utilization 

Environment 

Range 

Nominal 

Loading 

Principle  of  Operation 
Software 

Implementation  Languages 
Complexity 
Lines  of  Code 
Nesting 
Numbers 
Levels 
Type 

Specifications 

Switches 

Number  Used 
Type 

Manual  Automatic 
Actuating  Mechanism 
Mechanical 
Electromagnetic 
Throws 
Poles 
Action 

Momentary  Push  Button 
Push  Button 
Toggle 
Knife 
Rotary 
Construction 
Materials 
Packaging 
Ratings 
Voltage 
Current 
Frequency 
Utilization 
Voltage 
Current 
Frequency 
Mechanical 
Frequency 
Duty  Cycle 
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Transforaers 


SYSTEM  CIORACTERISTICS 


Nunber  Used 
Construction 
Core 

Area  Product 
Material 

Composition  (Laminated. 

Power.  Tape  Wound,  etc.) 
Primary  Coll 
Turns 
Wire  Type 
Taps 

Secondary  Coll 
Turns 
Wire  Type 
Taps 

Utilization 
Primary  Voltage 
Nominal 
Peak 

Frequencies 

Nqminal 

Maximum 

Secondary  Loading 
Nominal 
Peak 

"Transistors 
Number  Used 

Family  (Bipolar  NPN.  Bipolar  NPN. 

N-Channel.  JFETS.  etc.) 
Application  (Linear.  Digital. 

Power) 

Ratings 
Bandwidth 
Transconductance 
Capacitances 
Peak  Voltages 
Peak  Currents 
Usage 

Power  Dissipated 
Frequency  Range 
Peak  Voltages 
Peak  Currents 
Cooling  Technique 
Mounting  Technique 
Mechanical 

Long  Lead  -  Floating 
Short  Head 
Thermal  Mount 


Construction 

Packaging 
Materials 
Dimensions 
Ccxnponent  Density 

Functions 

Number  of  Discrete  Functions 
Usage 

Frequency 
Duty  Cycle 

Fault  Tolerant  Characteristics 

Inputs 

Outputs 

Interfaces 

Electrical 

Inputs 

Outputs 

Mechanical 

Mounting 

Operational 

Operator 

Difficulty  of  Use 
Operator  Qualifications 

Requirements 

Power 

Minimum 

Maximum 

Nominal 

Voltages 

Cooling 

Topology 

Subsystems 

Number 

Types 

Interdependencies 
Sneak  Circuits 

Fault  Tolerant  Characteristics 


Usage 


CCTitamination  (continued) 


Frequency 
Duty  Cycle 


TESTABILITY  CHARACTERISTICS 

Built  In  Test 

Exists  (Y/N)  Unit 
Hosted  in  Under  Test  (Y/N) 
Tests 
Number 

Information  Captured 
Rel iabi 1 1 ty /Conf Idence 
Interdependencies 
Construction 
Components 
BIT  Topology 

External  Testing 

Tests 

Number 

Information  Captured 
Rel iabi 1 i ty /Conf idence 
Interdependencies 
Construction 
Components 
Test  System  Topology 
Operational 
Difficulty  of  Use 
Operator  Qualifications 


ENVIRONMENT 

Contamination 

Particulates 

Types 

Concentrations 

Nominal 

Extremes 

Moisture 

Nominal 

Extremes 

Lubricants 

Types 

Concentrations 


Nominal 

Extremes 

Fuel 

Types 

C(»)centrations 

Nominal 

Extremes 

Electromagnetic 

Electromagnetic  Interference 
Spectra 

Nominal  Magnitude 
Maximum  Magnitude 
Power  Surges 

Frequency  of  Occurrence 
Magnitude  (Max) 

Static  Discharge 

Frequency  of  Occurrence 
Magnitude  (Max) 

Transient  Spectra 
Nominal  Magnitude 
Maximum  Magnitude 

Mechanical 

Vibration  Spectra 
Nominal  Magnitude 
Maximum  Magnitude 
Shock 

Frequency  of  Occurrence 
Magnitude  (Max) 

Acoustic  Spectra 
Nominal  Magnitude 
Maximum  Magnitude 
Pressure 
Nominal 
Extremes 

Maximum  Rate  of  Change 
Thermal 

Ambient  Temperature 
Nominal 
Extremes 
Cycling 
Shock 

Frequency  of  Occurrence 
Magnitudes 


MISCELLANEOUS 


Acquisition  Cost 
BIT/TE  Cost 
Degree  of  Burn-In 
Demonstration  Results 
Design  Life 
Equipment  Type 

Acceptance  Testing  Complexity 
FMEA  Requirement 
Lines  of  Code 
Operating  Code 


BIT  Code 
Lot  Size 

Maintenance  Concept 
Maintenance  Accessibility 
Manufacturer 
Maturity 

Off-the-shelf  Percent 
Pages  of  Maintenance 
Documentation 
Quantity  Produced 
R&D  Cost 

Testability  Requirement 
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ARINC  RESEARCH 
TESTABILITY  PARAMETER 
WORKSHEET 
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LRU  (MNEMONIC/NAME)  ; 
USED  IN  WEAPON  SYSTEM: 
ANALYST(S): 


DATE: 


INFORMATION  SOURCES: 


ATTACH  LRU  FUNCTIONAL  BLOCK  DIAGRAI 


worksheet  page  1 


TESTABILITY  PARAMETER  WORKSHEET 


ENVIRONMENTAL  FACTORS  CHECKJ.IST 


THERMAL  CHARACTIRISTICS 


EXTREMES  I  YES 


CYCLING  YES  ( 


SHOCK 


YES 

( _ I 

YES 

[ _ 1 

YES 

I _ 1 

YES  [  ] 

PARTICULATES 


MOISTURE 


LUBRICANTS 


MECHANICAL  CHARACTERISTICS 


VIBRATION  YES  [ 


SHOCK  I  YES  ( 


ACOUSTIC  I  YES  [ 


YES  [ 


NO  [ 


NO 


NO 


NO  [ 


NO 


MO  [ 


NO  [ 


NO  ( 


EMI 


SURGE 


STATIC 


YES  ( 


NO  [ 


wor)csh«et  page  4 


TESTABILITY  PARAMETER  WORKSHEET 


worksheet  page  6 
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APPENDIX  C 


VARIABLES  USED  FOR  REGRESSION  ANALYSES 


TABLE  C-1 


C«}  INITIAL  DESIGN  VARIABLES 


Parts* 

(NoriMillzad  ty 


No.  of  SRUs) 

External* 

Topological** 

Environment* 

Total  Parts 

Input/Output 

Signals 

10  Branches 
Functions  *  Nodes 

TOPI 

Thermal 

Resistors 

Length 

10 

T0P2 

Contaminants 

Potentloneters 

Height 

Functicxis 

Mechanical 

Capacitors 

Width 

Cross  Count  . 

T0P3 

Electrical 

Functions 

ICS 

Weight 

Cross  Count  > 

T0P4 

Transistors 

Power 

No.  of  SRUs 

Diodes 

External 

Connectors 

Inductors/ 

Transformers 

No.  SRU 

Relays 

Plane 

Switches 

BIT  (y/N) 

Indicators 

Pages  of 
Documentation 

Crystals 

Percent  Digital 

Interconnects 

*Taken  from  the  worksheets  as  delineated  in  Chapter  Three  and  given  in 
Appendix  B. 

**Derived  based  upon  analytical  work  given  in  Section  3. 4. 2. 2. 


C-3 


TABLE  C-2 

CND  VARIABLES  CREATED  PROH  INITIAL  VARIABLES 


Densities* 


PoMr  interactions** 


Voluna 

Density  -  l/volvsne 
Mrden  >  power/volume 
Prts<len  -  No.  parts/voluae 


Pwrden  •  power*parts  dwislty 
Thmpwr  •  poMer*(thenMil  ^  1) 

^rvol  -  power*volune 
Voverp  -  voluM/power 

Pwrthccn  *  poMer*lnterconnects* (thermal  *  1) 

0  If  pwr  <  150  watts 
PwrOl  -  1  If  pwr  >  150  watts 


Environmental  Interactlonsf 


Topological  Interact lonsff 


Thrmcom  «  (thermal  +  l)*lnterconnects 
Hechlo  >  (mechanical  l)*IO 
Mechcon  »  (mechanical  ■*-l>*lnterconnects 
Thmcroplx  <■  (thermal  *  l)*cmplx 


ictopl  «  ICs*TOPl 
Indtopl  •  lnd*TOPl 
Captopl  -  cap*TWl 


NlscellaneousS 

Basewt  «  1  per  no.  of  bases  in  data  collection 
Failure-  MA  -  CND 

Fir rate  >  No.  of  f allures/LRU  Operating  hours 


*Based  upon  analysis  of  accessibility  parameters  in  3. 4. 2.1  and  3. 4. 2. 7. 
**Control  variables. 

tBased  upon  analysis  of  environmental  factors  In  3. 4. 2. 3. 
ttBased  upon  analysis  of  topological  parameters  In  3. 4. 2. 2. 

SBased  upon  failure  frequency  parameters  In  3. 4. 2. 7. 
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TABLE  C-3 

CND  VARIABLES  SYNTHESIZED  BY  REGRESSION 


Thermal  Density* 

Thden6  *  thermal  +  6*prtsden 

ThdenS  «  thermal  +  5*prtsden 

Thdenm  =  prtsden* (thermal  +  1) 

Thdenl.5  »  thermal  +  1.5*prtsden 


Accessibility** 

Access  *  vol  +  500*cnnct 
where: 

Cnnct  B  small  ext  cnnctrs 
+  2*med  ext  cnnctrs 
+  3* large  ext  cnnctrs 


Parts  Complexltyt 


Sum2 

Sum3 

Suro4 

Capind 

Cmplx 

Transient 


[diodes  ^  l/lnductors]/No.  SRUs 

[resistors  *  transistors] /No.  SRUs 

[diodes  -  2*lnductors]/No.  SRUs 

[capacitors  -  2. 2* inductors] /No.  SRUs 

[ICs  +  10*lnductors  -  2.4*capacitors]/Nc.  SRUs 

[41*relays  +  2*capacitors  +  ICs  +2*reslstors 

-9* transistors] /No.  SRUs 


Topological  Complexitytt 

Tpcmplx  «  f (10. interconnects. SRU, parts) 
tcl  -  tc6  «  Intermediates  (topological) 

tc7  =  Interconnects*  (ICs  +  30*transistors  -  160*relays 

-  960*swltches) 

tc8  =  Interconnects*  (ICs  +  120*relays  460*switches 

-  40*transistors) 


*Bascd  upon  Initial  stepwise  linear  regressions  and  environ¬ 
mental  functions  In  3. 4. 2. 3. 

**Based  upon  initial  stepwise  linear  regressions  and  accessi¬ 
bility  functions  In  3. 4. 2.1  and  3. 4. 2. 7. 
fBased  upon  Initial  stepwise  linear  regressions  and  parts 
functions  In  3. 4. 2. 4.  3. 4. 2. 5.  and  3. 4. 3. 6. 
tfBased  upon  initial  stepwise  linear  regressions  and  topo¬ 
logical  functions  In  3. 4. 2. 2. 


TABLE  C-4 


IL  INITIAL  DESIGN  VARIABLES 


Parts* 

External* 

STAMP*  Measures** 

Number  of  SRUs 

Input/output 

Number  of  tests 

Total  parts 

Length 

Number  of  components 

Number  of  resistors 

Height 

Number  of  input 

Number  of  potentiometers 

Width 

signals 

Number  of  capacitors 
Number  of  ICs 

Number  of  transistors 
Number  of  diodes 

Number  of  inductors/ 
transformers 

Number  of  switches 

Number  of  indicators 
Number  of  crystals 

Number  of  filters 

Number  of  Interconnects 

Weight 

Power 

External  connectors 
(Small.  Medium. 
Large. 

Test  leverage 

*Based  upon  worksheets  In  Appendix  B. 

**Based  upon  functionalities  developed  in  3.4.3 


TABLE  C-5 


'•i  '  t  '■ 

IL  COMBINED  AND  SYNTHESIZED  VAKIaBL'i^^ 


Combination  of  Initial  Variables 


Densities*  Miscellaneous** 


Volume  test.cmp  «  tests/components 

Parts  density  input. cmp  *  inputs/components 

test.prt  *  tests/parts 
comp.tst  »  components/tests 
lo.cmp  io/components 


Variables  Synthesized  by  Regression 


Topological  Measures** 


■interconnects* (ICs  +  150*relays  -  17*transistors) 
,prt  ■  TC/parts 
•tst  ■  TC/tests 
test.tc  «  tests/TC 
testtc.crap  *  tests*TC/coroponents 


*Based  upon  accessibility  in  3. 4. 2.1  and  3. 4. 2. 7. 

**Based  upon  IL  functionalities  in  3.4.3  an’  topo¬ 
logical  parameters  in  3. 4. 2. 2.1  together  with 
initial  stepwise  linear  regressions. 


TABLE  C-6 


DP  COMBINED  AND  SYNTHESIZED  VARIABLES* 


tc2  »  Interconnects* (30*translstors-ICs  -  300relays) 


tsttc  ■  tests*TC 

testtc2  *  tests*tc2 

tc2sqd  »  {tc2)^ 

exptst.ctc  -  exp (test. ct2) 

tst  -  l.ct2  *  l/tst.ct2 

abstc2.tst  »  abs(tc2)/tests 

tlniin.r.st  «  tlmln/tests 

exp  -  tim.tst  *  exp(  -  tlialn.tst) 

exptst.att  =  exp (test. at t) 

InlO  *  In (10) 

ILFACTOR  *  (1  -  ID /components 
10. tst  ■  10/ tests 
OUTMEAS  »  10  -  3*inputs 
IninvFAT  =  ln(FAT‘ 

FATFACTOR  «  FAT"^/^ 


test.tc  ■  tests/TC 

test. etc  *  tests/components* tc2 

test.ct2  -  test.ctc/tc2 

2 

tsttc. emp  »  (tests*tc2  /components 
lntst.ct2  «  ln(tst.ct2) 
comp.tIO  »  components/ (tests* 10) 
exptlm.tst  «  exp(tlmln.tst) 
tst.att  -  tests/ («d5s(tc2)*tlmin) 
abstc2.tst  -  abs(tc2> /tests 
exp  -  10  “  exp(  -  10) 
lO.cmp  =  lO/components 
lOcmp  =  10* components 
exp  ILFACTOR  «  exp (ILFACTOR) 

FAT-n  =  FAT~”  where  n  is  a  constant 


*These  were  synthesized  based  on  the  initial  stepwise  linear  regressions 
and  all  of  the  functions  reported  in  Chapter  Three.  The  large  number  of 
synthesized  variables  points  to  the  weak  correlations  that  were  present 
with  no  apparent  dominance  by  any  ewnbination  of  terms. 


APPENDIX  D 


GLOSSARY  OF 
ACRONYMS  AND  1‘ERMS 


1.0  ACRONYMS 

AFTO 

Air  Force  Technical  Order 

ATE 

Automatic  Test  Equipment 

BIT 

Built-In  Test 

BITE 

Built-In  Test  Equipment 

CND 

Cannot  Duplicate 

DFT 

Design  for  Testability 

FAR 

False-Alarra  Rate 

FAT 

False-Alana  Tolerance 

FCND 

Fraction  of  Cannot  Duplicate 

FFA 

Fraction  of  False  Alarms 

FFD 

Fraction  of  Faults  ^Detected 

FFI 

Fraction  of  Faults  Isolated 

hofw-mal 

How-Malfunctioned 

IC 

Integrated  Circuits 

IL 

Isolation  Level 

10 

Input  and  Output  (Signals) 

LRU 

Line  Replaceable  Unit 

MA 

Maintenance  Action 

MADARS 

Malfunction  Analysis  Detection  and  Recording  System 

MDCS 

Maintenance  Data  Collection  f./stem 

MFLS 

Maintenance  Fault  Listing  Summary 

MTFI 

Mean  Time  to  Fault  Isolate 

MTTR 

Mean  Time  to  Repair 

D-J, 


NDP 

NFF 

NITS 

Nsn 


Nondetection  Percentage 
No  Fault  Found 
Not  Isolatable  This  Station 
Normal  System  Maintenance 


0/1  Organizational  and  Intermediate 

OPSREADY  Operations  Ready 


RTOK 


Retest -OK 


SRU 

STAMP* 

SUT 


Shop  Replaceable  Unit 

Systems  Testability  atKl  Maintenance  Program 
Subsystem  Under  Test 


TC 

TLMIN 

TO 


Topological  Complexity 
Theoretical  Minimum  Test  Leverage 
Technical  Order 


2.0  TERMS 

Abnormal  Fault  Isolation  -  Techniques  used  to  identify  the  cause  of 
subsystem  under  test  (SUT)  failure  by  means  other  than  normal  system 
maintenance  procedures:  for  example.  (1)  removal  of  multiple  replaceable 
units  and  (2)  shotgun  removal  of  replaceable  units  until  the  SUT  is 
operational. 


AFTO-349  -  Air  Force  Maintenance  Data  Collection  Record. 


Attribute  -  A  hypothesized  inherent  aspect  of  a  system  that  may  or  may  not 
be  observable  or  computable,  but  is  inferred  to  describe  some  aspect  of 
the  system. 

Bad  Actor  -  Any  SUT  with  repeat  failure  indications  that  carmot  be  dupli¬ 
cated  or  verified  during  normal  system  maintenance.  Bad  actors  may  be 
"recognized"  over  a  period  of  time  or  may  be  "Indicated"  by  outside 
sources.  Bad  actors  may  be  generic  —  line  replaceable  unit  (LRU)  type  — 
or  specific  —  a  given  serial  number. 

Built  In  Test  (BIT)  -  A  test  subsystem  that  is  a  physical  part  of  a  host 
system  whose  purpose  is  fault  detection.  For  a  given  LRU  within  the 
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system.  BIT  may  be  fully  contained  within  the  LRU  or  the  LRU  may  be  tested 
by  systemwide  BIT.  For  purposes  of  this  report,  either  case  may  exist. 

Cannot  Duplicate  (CND)  or  No  Fault  Found  (NFF)  -  There  is  a  prior  indica¬ 
tion  of  failure,  and  the  failure  cannot  be  duplicated  by  maintenance. 

Characteristic  -  attribute. 

Failure  Mode  Component  -  A  failure  mode,  piece  of  hardware,  or  other 
functional  entity  that  could  be  concluded  as  the  cause  of  a  syst^  anomaly. 

Depot -Level  Maintenance  -  Performs  piece  part  repair  within  an  SRU. 

False-Alarm  Rate  (FAR)  -  The  rate  of  occurrence  of  false  alarms,  typically 
computed  as  the  time-normalized  sum  of  false  alarms,  where  the  time 
normalized  is  either  calendar  or  operating  hours. 

Fault  Isolation  -  The  method  by  which  failures  are  located  as  a  first  step 
in  the  repair  process. 

Functional  Input  Signal  -  All  input  representing  a  function.  For  example, 
a  bus  line  with  several  signals  but  only  one  function  would  be  considered 
as  one  functional  input  signal. 

Inherent  Testability  -  A  testability  measure  that  is  dependent  only  on 
hardware  design  and  is  independent  of  test  stimulus  and  response  data 
(MIL-STD-2165  definition). 

Input  Signal  -  Any  active  electrical  signal  including  power  and  ground  but 
not  including  spares. 

Intermediate-Level  Maintenance  (I-Level)  -  Performs  shop-replaceable  unit 
(SRU)  replacement  repair  at  shop  level. 
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Intermittent  Failure  -  Transient*  failure  mode  of  the  SUT  that  Is  not 
reproducible  by  using  normal  system'  maintenance.  The  failure  may  or  may 
not  be  present  during  maintenance  rhecks.  Repeat  transient  failures  may 
label  an  SUT  as  a  "bad  actor"  and  r  .  uit  In  replacement  without  mal.’^tenance 
verification  of  the  fault. 

Line-Replaceable  Unit  (LRU)  -  Organizational-level  repair  unit  consisting 
of  a  collection  of  parts  packaged  for  replacement  at  the  flight  line. 

Maintenance  System  Fault  Detection  -  An  Indication  Is  provided  by  normal 
system  maintenance  that  the  SUT  Is  not  functioning  properly  because  of  a 
real  failure  within  the  SUT. 

Maintenance  System  Fault  Isolation  -  Ability  to  Identify  all  failed 
replaceable  units  within  the  SUT  using  normal  system  maintenance.  Fault 
Isolation  may  be  subdivided  Into  the  following  categories:  BIT  fault 
isolation,  automatic  fault  Isolation,  and  manual  or  semiautomatic  fault 
isolation. 

Measure  -  An  Individual  quantity  that  can  be  obtained  by  observation, 
calculation,  or  other  direct  means. 

Nonrelevant  Event  -  Any  fault  indication  that  does  not  result  in  a  mainte¬ 
nance  action. 

Normal  System  Maintenance  (NSN)  -  Techniques  that  are  specified  as  standard 
operating  procedures  for  use  of  BIT.  automatic  test  equipment  (ATE)  semi¬ 
automatic.  or  documented  manual  detection  and  troubleshooting  for  a  given 
system  under  test.  This  includes  regular  calendar  checks  and  normal 
go-checks.  It  Is  sometimes  called  "defined  means." 

Not  Isolatable  This  Station  (NITS)  -  Normal  or  abnormal  fault-isolation 
procedures  cannot  determine  the  cause  of  fault  in  the  SUT.  Maintenance 
concept  at  0-level  may  be  to  ship  the  SUT  to  another  level. 


Operational  Maintenance  Level  (0-Level).-  The  level  of  maintenance 
concerned  with  the  operational  readiness  of  the  weapon  system  and  generally 
consists  of  repair  action  by  LRU  replacement. 

Organizational -Level  Maintenance  -  Performs  LRU  replacement  and  provides 
point  of  readiness  for  operation. 

Parameter  -  A  variable  that  is  fixed  in  value. 

Redball  -  A  last-ditch  effort  to  save  a  mission  idien  the  scheduled  aircraft 
is  faulty.  TAG  and  SAC  call  this  "Redball."  and  MAC  calls  it  "Red  Streak." 
It  also  has  been  referred  to  as  "Blue  Streak"  by  SAC. 

Retest-OK  (RTOK)  -  A  replaceable  unit  is  removed,  but  no  failure  is 
discovered  at  subsequent  levels  of  maintenance.  A  RTOK  does  not  auto¬ 
matically  imply  that  no  failure  exists. 

Shotgun  Maintenance  -  Random  removal  and  replacement  of  LRUs  in  order  to 
find  and  repair  faults. 

Shop-Replaceable  Unit  (SRU)  -  Intermediate-level  repair  unit  consisting  of 
a  collection  of  parts  packaged  for  replacement  at  the  shop  level. 

Subsystem  False  Alarm  -  A  failure  Indication  in  a  subsystem  when  there  is 
no  failure  in  the  system. 

Subsystem  Improper  Fault  Detection  -  Fault  is  within  the  subsystem  other 
than  the  one  in  vrtiich  detection  occurs. 

Subsystem  Improper  Fault  Isolation  -  All  but  not  only  failed  units  are 
isolated. 

Subsystem  Proper  Fault  Detection  -  Fault  is  within  the  subsystem  in  which 
detection  occurs. 
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Subsystem  Proper  Fault  Isolation  -  Only  and  all  failed  units  are  Isolated. 


Subsystem  Under  Test  (SUT)  -  All  of  the  equipment  associated  with  a 
subsystem.  Including  BITE  but  excluding  test  equipment  that  Is  not 
physically  attached  during  normal  operation. 

Surrogate  -  A  measure,  variable,  attribute,  or  parameter  that  Is  used  as  a 
substitute  for  another  measure,  variable,  attribute,  or  parameter  because 
It  Is  either  easily  available,  a  good  estimate.  Inexpensive  to  obtain,  or 
all  of  the  above. 

System  False  Alarm  -  Normal  system  maintenance  Indicates  a  failure  In  the 
SUT  when  there  Is  no  failure  present. 

System  Go-Check  -  Normal  maintenance  procedures  used  to  verify  that  SUT  Is 
functioning  properly. 

System  Improper  Fault  Isolation  -  All  but  not  only  failed  units  are 
Isolated. 

System  Proper  Fault  Isolation  -  Only  and  all  failed  units  are  isolated. 

Test  -  An  individual  stimulus  response  vdiere  the  stimulus  may  or  may  not 
be  present  in  the  system.  A  fault  isolation  (test  procedure)  will  consist 
of  many  such  tests. 

Testability  -  A  design  characteristic  that  allows  the  status  (operable, 
Inoperable,  or  degraded)  of  an  Item  to  be  determined  and  the  isolation  of 
faults  within  the  item  to  be  performed  in  a  timely  manner  (MIL-STD-2165 
definition) . 

Variable  -  A  numeric  quantity  which  is  synthesized  by  combinations  of 
measures,  attributes,  or  other  properties. 
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